Emergency Response for Flooding Events
Task description
The Multimedia Satellite Task 2019 is offering three subtasks on flood detection, plus one exploratory subtask.
Image-based News Topic Disambiguation: In this subtask, participants receive links to a set of images that appeared in online news articles (English). They are asked to build a binary image classifier that predicts whether or not the topic of the article in which each image appeared was a water-related natural-disaster event. All of the news articles in the data set contain a flood-related keyword, e.g., “flood”, but their topics are ambiguous. For example, a news article might mention a “flood of flowers”, without being an article on the topic of natural-disaster flooding.
Multimodal Flood Level Estimation from News: In this task, participants receive a set of links to online news articles (English) and the accompanying images. The set has been filtered to include only news articles for which the accompanying image depicts at least one person. Participants are asked to build a binary classifier that predicts whether or not the image contains at least one person standing in water above the knee. Participants can use image-features only, but the task encourages a combination of image and text features, and even use of satellite imagery.
City-centered satellite sequences: In this task, participants receive a set of sequences of satellite images that depict a certain city over a certain length of time. They are required to create a binary classifier that determines whether or not there was a flooding event ongoing in that city at that time. Because this is the first year we work with sequences of satellite images, the data will be balanced so that the prior probability of the image sequence depicting a flooding event is 50%. This design decision will allow us to better understand the task. Challenges of the task include cloud cover, and ground-level changes with non-flood causes.
Fake/Non-Relevant Social Images Detection Task (Experimental): Official news sources provide high-quality, trustworthy coverage of flooding events. However, users posting information on social media can provide faster more detailed information on the evolution of a flooding event, and of its background. Unfortunately, social media sources contain a great detail of material that is intentionally misleading, or that is not relevant to the evolution of a flooding event. Participants will receive a set of images that they are required to classify as either relevant or non-relevant to a particular flooding event.
As always, we encourage task participants to release their code in order to support reproducibility of the results.
Target group
The task is of interest to researchers in the areas of online news, social media, multimedia analysis, multimedia information retrieval, flood-related analysis, water management, computer vision, remote sensing, and satellite image processing. We additionally encourage researchers focusing specifically on domains of disaster response, emergency response, and situational awareness to participate in the challenge.
Data
The Multimedia Satellite Task 2019 will issue three data sets:
- The news data set contains links to English-language news articles that include both text and images. The data is collected for a set of African cities.
- The satellite image data set contains sequences of satellite images from the same African cities. We plan to incorporate satellite images from the WorldView3 satellite of DigitalGlobe (0,3 m resolution). Additional metadata such as rainfall and climate predictions are provided as additional meta-data for each satellite image.
- The social media task will use a set of links to social media.
Ground truth and evaluation
- Image-based News Topic Disambiguation: The ground truth will be created by a text classifier based on the output of an advanced natural language processing tool that extracts references to specific events. In effect, this task will test whether image processing can supply the same information as state-of-the-art text analysis. Main metric will be ROC AUC, but the task may explore other metrics as well.
- Multimodal Flood Level Estimation from News: The ground truth will be created by human annotators working on a crowdsourcing platform. The annotators will be presented with the images, and ask whether the image depicts at least one standing person who is standing in water deep enough to reach over their knees (at least one knee is under water). Children are included in the definition of people, although they are shorter. Main metric will be ROC AUC, but the task may explore other metrics as well.
- City-centered satellite sequences: The ground truth will be created using the newspaper articles to find flood events. Negative examples will be chosen randomly. Main metric will be ROC AUC, but the task may explore other metrics as well.
Recommended reading
Analysis of Disaster Events in Social Media
[1] Brouwer, T., Eilander, D., Van Loenen, A., Booij, M. J., Wijnberg, K. M., Verkade, J. S., & Wagemaker,J. (2017). Probabilistic flood extent estimates from social media flood observations. Natural hazards and earth system sciences, 17(5), 735.
[2] Peters, R., and Albuquerque, J. P. D. (2015). “Investigating images as indicators for relevant social media messages in disaster management,”in The 12th International Conference on Information Systems for Crisis Response and Management (Kristiansand, Norway).
[3] Lagerstrom, Ryan, et al. ”Image classification to support emergency situation awareness.” Frontiers in Robotics and AI 3 (2016): 54.
[4] Schnebele, Emily, et al. “Road assessment after flood events using non-authoritative data.” Natural Hazards and Earth System Sciences, 2014.
[5] Middleton, Stuart, et al. “Real-Time Crisis Mapping of Natural Disasters Using Social Media.” IEEE Intelligent Systems, 2014.
[6] Schnebele, Emily, et al. “Real time estimation of the Calgary floods using limited remote sensing data.” Water 6.2 (2014): 381-398.
[7] Muhammad Imran, Carlos Castillo, Ji Lucas, Patrick Meier, and Sarah Vieweg: AIDR: Artificial In- telligence for Disaster Response. International Conference on World Wide Web (WWW), 2014. Seoul
Analysis of Disaster Events in Satellite Imagery
[1] Chaouch, Naira, et al. “A synergetic use of satellite imagery from SAR and optical sensors to improve coastal flood mapping in the Gulf of Mexico.” Hydrological Processes 26.11 (2012): 1617-1628.
[2] Klemas, Victor. “Remote sensing of floods and flood-prone areas: an overview.” Journal of Coastal Research 31.4 (2014): 1005-1013.
[3] Ogashawara, Igor, Marcelo Pedroso Curtarelli, and Celso M. Ferreira. “The use of optical remote sensing for mapping flooded areas.” International Journal of Engineering Research and Application 3.5 (2013): 1-5.
[4] Ticehurst, C. J., P. Dyce, and J. P. Guerschman. “Using passive microwave and optical remote sensing to monitor flood inundation in support of hydrologic modelling.” Interfacing modelling and simulation with mathematical and computational sciences, 18th World IMACS/MODSIM Congress. 2009.
We recommend also having a look at the past year’s task papers in the MediaEval Proceedings:
Guillaume Gravier et al. (eds.) 2017. Proceedings of the MediaEval 2017 Workshop, Dublin, Ireland, Sept. 13-15, 2017.
Martha Larson, et al. (eds.) 2018. Proceedings of the MediaEval 2017 Workshop, Sophia Antipolis, France, Oct. 29-31, 2018.
Task organizers
Benjamin Bischke, German Research Center for Artificial Intelligence (DFKI), Germany benjamin.bischke @ dfki.de
Patrick Helber, German Research Center for Artificial Intelligence (DFKI), Germany
Zhengyu Zhao, Radboud University, Netherlands
Tom Brouwer, Floodtags, Netherlands
Erkan Basar, Radboud University and FloodTags, Netherlands
Konstantin Pogorelov, Simula Research Laboratory, Norway
Task auxiliaries
Jens de Bruijn, Floodtags, Netherlands
Martha Larson, Radboud University, Netherlands
Task schedule
Development data release: 15 May
Test data release: 30 May
Runs due: 20 September
Results returned: 23 September
Working Notes paper due: 30 September
MediaEval 2019 Workshop (in France, near Nice): 27-29 October 2019
