Bharath Hariharan

I am an assistant professor in Computer Science at Cornell University. I work on computer vision and machine learning, in particular on important problems that defy the "Big Data" label. I enjoy problems that require marrying advances in machine learning with insights from computer vision, geometry and domain-specific knowledge. Currently, my group is working on building systems that can learn about tens of thousands of visual concepts with very little or no supervision, produce rich and detailed outputs such as precise 3D shape, and reason about the world and communicate this reasoning to humans. A sampling of the research problems my group works on is presented below; an exhaustive list of publications is available on scholar.

My work has been recognized with an NSF CAREER award and a PAMI Young Researcher Award.

My CV is here and my research statement is here.

Note to prospective PhD students: Admissions at Cornell are done through a committee. If you are interested in working with me, please directly apply through the application website and mention my name

Assistant Professor
311 Gates Hall
Cornell University
bharathh-AT-cs-DOT-cornell-DOT-edu

Teaching

CS6670 (Graduate computer vision): Fall 2021, Fall 2019, Fall 2018, Fall 2017
CS4670 / 5670 (Undergraduate computer vision): Spring 2021, Spring 2020, Spring 2019,Spring 2018

PhD students

Yihong Sun
Aditya Chetan
Cheng Perng Phoo
Luming Tang
Utkarsh Mall (with Kavita Bala)
Chinasa Okolo (with Nicola Dell)
Qianqian Wang (with Noah Snavely)
Ruojin Cai (with Noah Snavely)
Guandao Yang (with Serge Belongie)
Yurong You (with Kilian Weinberger)

Former PhD students

Yan Wang → Waymo
Bram Wallace → Salesforce Research
Davis Wertheimer → IBM Research

Research

Recognition with minimal labels

Deep learning and ConvNets revolutionized visual recognition, but require large labeled datasets for training. This is a problem in new domains like satellite imagery, in expert applications like fine-grained recognition, and in "open-world" settings like robotics where the space of possible classes is not known a priori. We are designing new classes of recognition systems that can be trained with very few labeled examples and can even discover classes on their own. The key insight is to look beyond the available data, leveraging domain knowledge and visual learning that transcends domains. Funding: This work is funded by an NSF CAREER award, DARPA and IARPA.
Papers

Learning to reconstruct and synthesize 3D

Humans can not just recognize objects but also reconstruct 3D objects. We can reason about 3D even when we can only see one view, or a few sparse views. For computer vision systems to have this ability, they must reason not just about the well-explored geometry of perspective projection, but also about priors about scenes and shapes. Combining mathematical constraints from geometry with the data-driven priors provided by machine learning is an open research question.
Papers

Recognition in 3D

With advances in 3D reconstruction and recognition, vision is now being deployed in a variety of robotics applications, including self-driving cars. These robots have multiple cameras and LiDAR sensors, and require precise 3D location estimates for control. We are bringing insights from recognition, limited-label learning and 3D reconstruction/synthesis to perception in 3D . Funding:This work is funded by NSF.
Papers

Recent papers

New Pre-prints

Emergent Correspondence from Image Diffusion
Luming Tang*, Menglin Jia*, Qianqian Wang*, Cheng Phoo, Bharath Hariharan
In Submitted, 2023
TLDR : Semantic and geometric correspondences can be extracted from diffusion models without any further training
pdf bibtex
```
@inproceedings{Tang2023Emergent,
title = {Emergent Correspondence from Image Diffusion},
author = {Luming Tang and  Menglin Jia and  Qianqian Wang and  Cheng Phoo and  Bharath Hariharan},
year = {2023},
booktitle = {Submitted}
}
```

Recent publications

Tracking Everything Everywhere All at Once
Qianqian Wang, Yen-Yu Chang, Ruojin Cai, Zhengqi Li, Bharath Hariharan, Aleksander Holynski, Noah Snavely
In ICCV, 2023
TLDR : A neural field approach to estimating long-term correspondences even through occlusions
pdf bibtex

@inproceedings{Wang2023Tracking,
title = {Tracking Everything Everywhere All at Once},
author = {Qianqian Wang and  Yen-Yu Chang and  Ruojin Cai and  Zhengqi Li and  Bharath Hariharan and  Aleksander Holynski and  Noah Snavely},
year = {2023},
booktitle = {ICCV}
}

Distilling from Similar Tasks for Transfer Learning on a Budget
Kenneth Borup, Cheng Perng Phoo, Bharath Hariharan
In ICCV, 2023
TLDR : We train models for new domains with limited data and limited compute by identifying pre-training domains and distilling from them
pdf bibtex
```
@inproceedings{Borup2023Distilling,
title = {Distilling from Similar Tasks for Transfer Learning on a Budget},
author = {Kenneth Borup and  Cheng Perng Phoo and  Bharath Hariharan},
year = {2023},
booktitle = {ICCV}
}
```
Doppelgangers: Learning to Disambiguate Images of Similar Structures
Ruojin Cai, Joseph Tung, Qianqian Wang, Hadar Averbuch-Elor, Bharath Hariharan, Noah Snavely
In ICCV, 2023
TLDR : We learn to classify whether an image pair in an SfM pipeline captures the same structure, or similar but distinct structures
pdf bibtex
```
@inproceedings{Cai2023Doppelgangers,
title = {Doppelgangers: Learning to Disambiguate Images of Similar Structures},
author = {Ruojin Cai and  Joseph Tung and  Qianqian Wang and  Hadar Averbuch-Elor and  Bharath Hariharan and  Noah Snavely},
year = {2023},
booktitle = {ICCV}
}
```
Change-Aware Sampling and Contrastive Learning for Satellite Images
Utkarsh Mall, Bharath Hariharan, Kavita Bala
In CVPR, 2023
TLDR : A self-supervised learning approach for satellite images that pushes apart images taken from the same location but over long time spans
pdf bibtex
```
@inproceedings{Mall2023Change,
title = {Change-Aware Sampling and Contrastive Learning for Satellite Images},
author = {Utkarsh Mall and  Bharath Hariharan and  Kavita Bala},
year = {2023},
booktitle = {CVPR}
}
```
Change Event Dataset for Discovery from Spatio-temporal Remote Sensing Imagery
Utkarsh Mall, Bharath Hariharan, Kavita Bala
In NeurIPS (Datasets and Benchmarks track), 2022
TLDR : A new benchmark on discovering meaningful multi-step change events from satellite images with few/no labels
pdf bibtex
```
@inproceedings{Mall2022Change,
title = {Change Event Dataset for Discovery from Spatio-temporal Remote Sensing Imagery},
author = {Utkarsh Mall and  Bharath Hariharan and  Kavita Bala},
year = {2022},
booktitle = {NeurIPS (Datasets and Benchmarks track)}
}
```
Polynomial Neural Fields for Subband Decomposition and Manipulation
Guandao Yang*, Sagie Benaim*, Varun Jampani, Kyle Genova, Jonathan T. Barron, Thomas Funkhouser, Bharath Hariharan, Serge Belongie
In NeurIPS, 2022
TLDR : A new architecture for neural fields that explicitly decomposes signals into Fourier and other bases
pdf bibtex
```
@inproceedings{Yang2022Polynomial,
title = {Polynomial Neural Fields for Subband Decomposition and Manipulation},
author = {Guandao Yang and  Sagie Benaim and  Varun Jampani and  Kyle Genova and  Jonathan T. Barron and  Thomas Funkhouser and  Bharath Hariharan and  Serge Belongie},
year = {2022},
booktitle = {NeurIPS}
}
```
Unsupervised Adaptation from Repeated Traversals for Autonomous Driving
Yurong You*, Cheng Perng Phoo*, Katie Z Luo*, Travis Zhang, Wei-Lun Chao, Bharath Hariharan, Mark Campbell, Kilian Q. Weinberger
In NeurIPS, 2022
TLDR : Data captured from repeated traversals of a scene can be used to adapt 3D object detectors to new domains without supervision
pdf bibtex
```
@inproceedings{You2022Unsupervised,
title = {Unsupervised Adaptation from Repeated Traversals for Autonomous Driving},
author = {Yurong You and  Cheng Perng Phoo and  Katie Z Luo and  Travis Zhang and  Wei-Lun Chao and  Bharath Hariharan and  Mark Campbell and  Kilian Q. Weinberger},
year = {2022},
booktitle = {NeurIPS}
}
```

Visual Prompt Tuning
Menglin Jia*, Luming Tang*, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, Ser-Nam Lim
In ECCV, 2022
TLDR : Learning a prompt is the best way to transfer vision transformers to new tasks
pdf bibtex

@inproceedings{Jia2022Visual,
title = {Visual Prompt Tuning},
author = {Menglin Jia and  Luming Tang and  Bor-Chun Chen and  Claire Cardie and  Serge Belongie and  Bharath Hariharan and  Ser-Nam Lim},
year = {2022},
booktitle = {ECCV}
}

Exploiting Playbacks in Unsupervised Domain Adaptation for 3D Object Detection
Yurong You*, Carlos Andres Diaz-Ruiz*, Yan Wang, Wei-Lun Chao, Bharath Hariharan, Mark Campbell, Kilian Weinberger
In ICRA, 2022
TLDR : Extrapolating tracks of detected objects yields good ground truth for adapting 3D detectors
pdf bibtex

@inproceedings{You2022Exploiting,
title = {Exploiting Playbacks in Unsupervised Domain Adaptation for 3D Object Detection},
author = {Yurong You and   Carlos Andres Diaz-Ruiz and  Yan Wang and  Wei-Lun Chao and  Bharath Hariharan and  Mark Campbell and  Kilian Weinberger},
year = {2022},
booktitle = {ICRA}
}

Learning to Detect Mobile Objects from LiDAR Scans Without Labels
Yurong You*, Katie Luo*, Cheng Perng Phoo, Wei-Lun Chao, Wen Sun, Bharath Hariharan, Mark Campbell, Kilian Weinberger
In CVPR, 2022
TLDR : Can we train a 3D object detector without labels by simply driving around?
pdf bibtex

@inproceedings{You2022Learning,
title = {Learning to Detect Mobile Objects from LiDAR Scans Without Labels},
author = {Yurong You and  Katie Luo and  Cheng Perng Phoo and  Wei-Lun Chao and  Wen Sun and  Bharath Hariharan and  Mark Campbell and  Kilian Weinberger},
year = {2022},
booktitle = {CVPR}
}

Hindsight is 20/20: Leveraging past traversals to aid 3D perception
Yurong You, Katie Luo, Xiangyu Chen, Junan Chen, Wei-Lun Chao, Wen Sun, Bharath Hariharan, Mark Campbell, Kilian Weinberger
In ICLR, 2022
TLDR : Can past traversals through a dynamic scene help 3D perception of the current scene?
pdf bibtex

@inproceedings{You2022Hindsight,
title = {Hindsight is 20/20: Leveraging past traversals to aid 3D perception},
author = {Yurong You and  Katie Luo and  Xiangyu Chen and  Junan Chen and  Wei-Lun Chao and  Wen Sun and  Bharath Hariharan and  Mark Campbell and  Kilian Weinberger},
year = {2022},
booktitle = {ICLR}
}

Bharath Hariharan I am an assistant professor in Computer Science at Cornell University. I work on computer vision and machine learning, in particular on important problems that defy the "Big Data" label. I enjoy problems that require marrying advances in machine learning with insights from computer vision, geometry and domain-specific knowledge. Currently, my group is working on building systems that can learn about tens of thousands of visual concepts with very little or no supervision, produce rich and detailed outputs such as precise 3D shape, and reason about the world and communicate this reasoning to humans. A sampling of the research problems my group works on is presented below; an exhaustive list of publications is available on scholar. My work has been recognized with an NSF CAREER award and a PAMI Young Researcher Award. My CV is here and my research statement is here. Note to prospective PhD students: Admissions at Cornell are done through a committee. If you are interested in working with me, please directly apply through the application website and mention my name	Assistant Professor 311 Gates Hall Cornell University bharathh-AT-cs-DOT-cornell-DOT-edu
Teaching CS6670 (Graduate computer vision): Fall 2021, Fall 2019, Fall 2018, Fall 2017 CS4670 / 5670 (Undergraduate computer vision): Spring 2021, Spring 2020, Spring 2019,Spring 2018
PhD students Yihong Sun Aditya Chetan Cheng Perng Phoo Luming Tang Utkarsh Mall (with Kavita Bala) Chinasa Okolo (with Nicola Dell) Qianqian Wang (with Noah Snavely) Ruojin Cai (with Noah Snavely) Guandao Yang (with Serge Belongie) Yurong You (with Kilian Weinberger) Former PhD students Yan Wang → Waymo Bram Wallace → Salesforce Research Davis Wertheimer → IBM Research
Research
Recognition with minimal labels Deep learning and ConvNets revolutionized visual recognition, but require large labeled datasets for training. This is a problem in new domains like satellite imagery, in expert applications like fine-grained recognition, and in "open-world" settings like robotics where the space of possible classes is not known a priori. We are designing new classes of recognition systems that can be trained with very few labeled examples and can even discover classes on their own. The key insight is to look beyond the available data, leveraging domain knowledge and visual learning that transcends domains. Funding: This work is funded by an NSF CAREER award, DARPA and IARPA. Papers
Learning to reconstruct and synthesize 3D Humans can not just recognize objects but also reconstruct 3D objects. We can reason about 3D even when we can only see one view, or a few sparse views. For computer vision systems to have this ability, they must reason not just about the well-explored geometry of perspective projection, but also about priors about scenes and shapes. Combining mathematical constraints from geometry with the data-driven priors provided by machine learning is an open research question. Papers
Recognition in 3D With advances in 3D reconstruction and recognition, vision is now being deployed in a variety of robotics applications, including self-driving cars. These robots have multiple cameras and LiDAR sensors, and require precise 3D location estimates for control. We are bringing insights from recognition, limited-label learning and 3D reconstruction/synthesis to perception in 3D . Funding:This work is funded by NSF. Papers
Recent papers
New Pre-prints
Emergent Correspondence from Image Diffusion Luming Tang, Menglin Jia, Qianqian Wang, Cheng Phoo, Bharath Hariharan In Submitted, 2023 TLDR* : Semantic and geometric correspondences can be extracted from diffusion models without any further training pdf bibtex @inproceedings{Tang2023Emergent, title = {Emergent Correspondence from Image Diffusion}, author = {Luming Tang and Menglin Jia and Qianqian Wang and Cheng Phoo and Bharath Hariharan}, year = {2023}, booktitle = {Submitted} }
Recent publications
Tracking Everything Everywhere All at Once Qianqian Wang, Yen-Yu Chang, Ruojin Cai, Zhengqi Li, Bharath Hariharan, Aleksander Holynski, Noah Snavely In ICCV, 2023 TLDR : A neural field approach to estimating long-term correspondences even through occlusions pdf bibtex @inproceedings{Wang2023Tracking, title = {Tracking Everything Everywhere All at Once}, author = {Qianqian Wang and Yen-Yu Chang and Ruojin Cai and Zhengqi Li and Bharath Hariharan and Aleksander Holynski and Noah Snavely}, year = {2023}, booktitle = {ICCV} } Distilling from Similar Tasks for Transfer Learning on a Budget Kenneth Borup, Cheng Perng Phoo, Bharath Hariharan In ICCV, 2023 TLDR : We train models for new domains with limited data and limited compute by identifying pre-training domains and distilling from them pdf bibtex @inproceedings{Borup2023Distilling, title = {Distilling from Similar Tasks for Transfer Learning on a Budget}, author = {Kenneth Borup and Cheng Perng Phoo and Bharath Hariharan}, year = {2023}, booktitle = {ICCV} } Doppelgangers: Learning to Disambiguate Images of Similar Structures Ruojin Cai, Joseph Tung, Qianqian Wang, Hadar Averbuch-Elor, Bharath Hariharan, Noah Snavely In ICCV, 2023 TLDR : We learn to classify whether an image pair in an SfM pipeline captures the same structure, or similar but distinct structures pdf bibtex @inproceedings{Cai2023Doppelgangers, title = {Doppelgangers: Learning to Disambiguate Images of Similar Structures}, author = {Ruojin Cai and Joseph Tung and Qianqian Wang and Hadar Averbuch-Elor and Bharath Hariharan and Noah Snavely}, year = {2023}, booktitle = {ICCV} } Change-Aware Sampling and Contrastive Learning for Satellite Images Utkarsh Mall, Bharath Hariharan, Kavita Bala In CVPR, 2023 TLDR : A self-supervised learning approach for satellite images that pushes apart images taken from the same location but over long time spans pdf bibtex @inproceedings{Mall2023Change, title = {Change-Aware Sampling and Contrastive Learning for Satellite Images}, author = {Utkarsh Mall and Bharath Hariharan and Kavita Bala}, year = {2023}, booktitle = {CVPR} } Change Event Dataset for Discovery from Spatio-temporal Remote Sensing Imagery Utkarsh Mall, Bharath Hariharan, Kavita Bala In NeurIPS (Datasets and Benchmarks track), 2022 TLDR : A new benchmark on discovering meaningful multi-step change events from satellite images with few/no labels pdf bibtex @inproceedings{Mall2022Change, title = {Change Event Dataset for Discovery from Spatio-temporal Remote Sensing Imagery}, author = {Utkarsh Mall and Bharath Hariharan and Kavita Bala}, year = {2022}, booktitle = {NeurIPS (Datasets and Benchmarks track)} } Polynomial Neural Fields for Subband Decomposition and Manipulation Guandao Yang, Sagie Benaim, Varun Jampani, Kyle Genova, Jonathan T. Barron, Thomas Funkhouser, Bharath Hariharan, Serge Belongie In NeurIPS, 2022 TLDR : A new architecture for neural fields that explicitly decomposes signals into Fourier and other bases pdf bibtex @inproceedings{Yang2022Polynomial, title = {Polynomial Neural Fields for Subband Decomposition and Manipulation}, author = {Guandao Yang and Sagie Benaim and Varun Jampani and Kyle Genova and Jonathan T. Barron and Thomas Funkhouser and Bharath Hariharan and Serge Belongie}, year = {2022}, booktitle = {NeurIPS} } Unsupervised Adaptation from Repeated Traversals for Autonomous Driving Yurong You, Cheng Perng Phoo, Katie Z Luo, Travis Zhang, Wei-Lun Chao, Bharath Hariharan, Mark Campbell, Kilian Q. Weinberger In NeurIPS, 2022 TLDR* : Data captured from repeated traversals of a scene can be used to adapt 3D object detectors to new domains without supervision pdf bibtex @inproceedings{You2022Unsupervised, title = {Unsupervised Adaptation from Repeated Traversals for Autonomous Driving}, author = {Yurong You and Cheng Perng Phoo and Katie Z Luo and Travis Zhang and Wei-Lun Chao and Bharath Hariharan and Mark Campbell and Kilian Q. Weinberger}, year = {2022}, booktitle = {NeurIPS} } Visual Prompt Tuning Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, Ser-Nam Lim In ECCV, 2022 TLDR : Learning a prompt is the best way to transfer vision transformers to new tasks pdf bibtex @inproceedings{Jia2022Visual, title = {Visual Prompt Tuning}, author = {Menglin Jia and Luming Tang and Bor-Chun Chen and Claire Cardie and Serge Belongie and Bharath Hariharan and Ser-Nam Lim}, year = {2022}, booktitle = {ECCV} } Exploiting Playbacks in Unsupervised Domain Adaptation for 3D Object Detection Yurong You, Carlos Andres Diaz-Ruiz, Yan Wang, Wei-Lun Chao, Bharath Hariharan, Mark Campbell, Kilian Weinberger In ICRA, 2022 TLDR : Extrapolating tracks of detected objects yields good ground truth for adapting 3D detectors pdf bibtex @inproceedings{You2022Exploiting, title = {Exploiting Playbacks in Unsupervised Domain Adaptation for 3D Object Detection}, author = {Yurong You and Carlos Andres Diaz-Ruiz and Yan Wang and Wei-Lun Chao and Bharath Hariharan and Mark Campbell and Kilian Weinberger}, year = {2022}, booktitle = {ICRA} } Learning to Detect Mobile Objects from LiDAR Scans Without Labels Yurong You, Katie Luo, Cheng Perng Phoo, Wei-Lun Chao, Wen Sun, Bharath Hariharan, Mark Campbell, Kilian Weinberger In CVPR, 2022 TLDR : Can we train a 3D object detector without labels by simply driving around? pdf bibtex @inproceedings{You2022Learning, title = {Learning to Detect Mobile Objects from LiDAR Scans Without Labels}, author = {Yurong You and Katie Luo and Cheng Perng Phoo and Wei-Lun Chao and Wen Sun and Bharath Hariharan and Mark Campbell and Kilian Weinberger}, year = {2022}, booktitle = {CVPR} } Hindsight is 20/20: Leveraging past traversals to aid 3D perception Yurong You, Katie Luo, Xiangyu Chen, Junan Chen, Wei-Lun Chao, Wen Sun, Bharath Hariharan, Mark Campbell, Kilian Weinberger In ICLR, 2022 TLDR : Can past traversals through a dynamic scene help 3D perception of the current scene? pdf bibtex @inproceedings{You2022Hindsight, title = {Hindsight is 20/20: Leveraging past traversals to aid 3D perception}, author = {Yurong You and Katie Luo and Xiangyu Chen and Junan Chen and Wei-Lun Chao and Wen Sun and Bharath Hariharan and Mark Campbell and Kilian Weinberger}, year = {2022}, booktitle = {ICLR} }