MONITRS: Multimodal Observations of Natural Incidents Through Remote Sensing
Shreelekha Revankar1, Utkarsh Mall2, Cheng Perng Phoo1, Kavita Bala1, Bharath Hariharan1
1Cornell University, 2Columbia University
Using news articles, we extract exact locations of disaster events and corresponding captions for event timelines. Our MONITRS dataset enables precise disaster monitoring, as shown in this Minnesota severe storm sequence. The May 27th image shows evidence of flooding with increased vegetation and darker water-saturated regions. Models finetuned with MONITRS correctly identify the temporal onset of the storm while baseline models fail to detect the initial evidence.
Abstract
Natural disasters cause devastating damage to communities and infrastructure every year. Effective disaster response is hampered by the difficulty of accessing affected areas during and after events. Remote sensing has allowed us to monitor natural disasters in a remote way. More recently there have been advances in computer vision and deep learning that help automate satellite imagery analysis, However, they remain limited by their narrow focus on specific disaster types, reliance on manual expert interpretation, and lack of datasets with sufficient temporal granularity or natural language annotations for tracking disaster progression. We present MONITRS, a novel multimodal dataset of ~10,000 FEMA disaster events with temporal satellite imagery with natural language annotations from news articles, accompanied by geotagged locations, and question-answer pairs. We demonstrate that fine-tuning existing MLLMs on our dataset yields significant performance improvements for disaster monitoring tasks, establishing a new benchmark for machine learning-assisted disaster response systems.
Paper Resources
Citation
@inproceedings{revankarmonitrs,
title={MONITRS: Multimodal Observations of Natural Incidents Through Remote Sensing},
author={Revankar, Shreelekha and Mall, Utkarsh and Phoo, Cheng Perng and Bala, Kavita and Hariharan, Bharath},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
year={2025}
}
Updates
Dataset Release
MONITRS dataset now available on Hugging Face!
Code Release
Code and implementation now public on GitHub!
Paper Acceptance
Paper accepted to NeurIPS 2025 Datasets and Benchmarks Track (Spotlight)
Paper Available
Paper now available on arXiv