Jul 10 (Wed) @ 2:00pm: "Building Efficient Vision Models for Ecological and Earth Observation Studies," Satish Kumar, ECE PhD Defense

Abstract

Large vision models for natural images, used in tasks such as detection, segmentation, and generation, owe their progress to the vast amounts of image and text data available on the internet. State-of-the-art models like SAM (trained on 11 million images with 1 billion annotations), Florence-2 (trained on 126 million images with 5.4 billion annotations), and GPT-4 (trained on 13 trillion tokens) highlight these advancements.  However, scientific challenges like earth observation and ecological studies, such as methane monitoring and tracking wildlife, present unique difficulties due to their complex nature and the scarcity of annotated data. These problems involve data from multiple sources and modalities. Acquiring these datasets is both expensive and time-consuming, with annotations often limited or absent. This dissertation addresses the critical question of how to build and train large vision models efficiently in such contexts, emphasizing the integration of domain-specific knowledge as inductive bias to optimize training efficiency. The effectiveness of this approach is demonstrated in applications ranging from methane detection to wildlife monitoring. 

In summary, this research establishes an effective methodology for integrating domain-specific knowledge into deep learning models, optimizing training processes and enhancing performance with limited annotated data. It provides significant insights and practical solutions for improving the applicability of vision transformer-based models across diverse domains.

Bio

Satish Kumar, is an ECE  PhD candidate advised by Professor B.S. Manjunath. His research work is focused on building large vision models for remote sensing problems, focusing on GHG emissions detection and wildlife detection from aerial imagery. Prior to joining the PhD program at UCSB in Winter 2019, he was lead researcher at Samsung R&D in South Korea from 2013-2017. He received his B.Tech from National Institute of Technology Kurukshetra in 2013. His PhD journey includes fellowships from the Smithsonian Institution, multiple internships and currently running his startup EyeClimate Inc. He was selected as a Schmidt Science Fellow in 2024 that  will enable him to explore the use of plants as biosensors to track environmental changes at scale.

Hosted by: Professor B.S. Manjunath

Submitted by: Satish Kumar <satishkumar@ucsb.edu>