The 2018 Multimedia Satellite Task:
Emergency Response for Flooding Events

Task description
The Multimedia Satellite Task requires participants to analyze social multimedia (Flickr, Twitter, Wikipedia) and/or satellite data for flooding events. The larger goal of this task is to combine the user view and the satellite view to obtain a comprehensive picture of such events. The user view is the semantically explicit, but spatially sparse, ground-level perspective represented by images in social media streams. The satellite view is the semantically vague, but spatially dense, top-down view captured by satellites.

Multi-modal event representation is of vital importance to achieve situational awareness and to provide support in emergency response, e.g., helping to coordinate rescuer efforts in large scale disasters. It is also important for studying disasters after they happen, and support planning that will prevent or mitigate the impact of future disasters. Examples of relevant disasters include natural disasters such as floods, fires, and earthquakes. The Multimedia Satellite Task 2018 continues to focus on flooding events, since, among high-impact natural disasters, flooding events represent the most common type of disaster worldwide. This year the task will look at “passibility”, namely whether or not it is possible to travel through a flooded region. “Passibility” is important for emergency response, and also is an area in which the information in social images has clear potential to complement the information in satellite images.

The main objective of this year's task is to quantify the impact of flooding events on infrastructure. The task involves two subtasks:

Flood classification for social multimedia:
The goal of this task is to retrieve all images from social media which provide direct evidence for passability of roads by conventional means (no boats, off-the-road vehicles, monster trucks, Hummer, Landrover, farm equipment). The objective is to design a system/algorithm/method that (in principle) given any collection of flood related multimedia images and their metadata (e.g., Twitter, Flickr, YFCC100M) is able to identify those images that (a) provide evidence for road passability and (b) to discriminate between images showing passable vs. non passable roads. In our context, road passability is related to the depicted the water level and the surrounding context. The correctness of retrieved images for the two classes (1) passable with evidence and (2) non passable with evidence will be evaluated with the F1-Score metric on the test set.

Flood detection in satellite images:
Participants receive data and are required to train classifiers. Fusion of satellite and social multimedia information is encouraged. The task moves forward the state of the art by concentrating on passibility, whether or not it is possible for a vehicle to pass a road. We hope that by looking at passibility we will open the door to future work that focuses on aspects of disasters that are important for people and for disaster relief, but are not generally studied by multimedia and remote-sensing researchers.

In 2018, we are encouraging task participants to release their code in order to support reproducibility of the results.

Target group
The task is of interest to researchers in the areas of social media, multimedia analysis, multimedia information retrieval, computer vision, remote sensing, and satellite image processing. We explicitly encourage researchers focusing specifically in domains of disaster response, emergency response and situational awareness to participate in the challenge.

The Multimedia Satellite Task 2018 introduces two novel datasets:
  • Data for “Flood classification for social multimedia”: The first subtask is based on the social media stream Twitter. The dataset contains a list of tweets with images for the three big Hurricane events of 2017: Harvey, Maria and Irma. The messages were collected on Twitter by filtering the tweets texts with the keywords (“flooding” and “flood”) during the time-frames of the three events. We will distribute a list of IDs for these tweets, along with tools to download the images and metadata of the tweet. In order to handle deleted tweets, we will additionally distribute a set of precomputed features for each image (classical visual features and CNN-based features). Remotely sensed information such as rainfall and climate predictions are provided as additional metadata for each image.
  • Data for “Flood detection in satellite images”: For the second subtask, satellite image patches of flooded areas for the above three events will be distributed. We plan to incorporate satellite images from different providers and resolutions into the dataset: (1) WorldView3 from DigitalGlobe (0,3 m resolution), (2) Planet from PlanetLabs (0,9-3,7m resolution) and publicly available satellites such as ESA's Sentinel 1 (RADAR) & 2 satellites and NASA's Landsat 7/8 satellites.

Ground truth and evaluation
The ground truth for the datasets of the two subtask is collected as follows:
  • For the “Flood classification for social multimedia” subtask, we plan to set up a crowdsourcing experiment to get labels for the collected twitter images via CrowdFlower. The evaluation metric will be F1.
  • For the “Flood detection in satellite images” subtask, we plan to use a custom tool. This tool allows to set the two coordinate points on satellite image and to select the corresponding label (passable/non passable). The evaluation metric will be F1.

Recommended reading
Analysis of Disaster Events in Social Media
[1] Brouwer, T., Eilander, D., Van Loenen, A., Booij, M. J., Wijnberg, K. M., Verkade, J. S., & Wagemaker,J. (2017). Probabilistic flood extent estimates from social media flood observations. Natural hazards and earth system sciences, 17(5), 735.
[2] Peters, R., and Albuquerque, J. P. D. (2015). “Investigating images as indicators for relevant social media messages in disaster management,”in The 12th International Conference on Information Systems for Crisis Response and Management (Kristiansand, Norway).
[3] Lagerstrom, Ryan, et al. ”Image classification to support emergency situation awareness.” Frontiers in Robotics and AI 3 (2016): 54.
[4] Yang, Yimin, et al. ”Hierarchical disaster image classification for situation report enhancement.” Information Reuse and Integration (IRI), 2011 IEEE International Conference on. IEEE, 2011.
[5] Schnebele, Emily, et al. “Road assessment after flood events using non-authoritative data.” Natural Hazards and Earth System Sciences, 2014

Analysis of Disaster Events in Satellite Imagery
[1] Lagerstrom, Ryan, et al. “Image classification to support emergency situation awareness.” Frontiers in Robotics and AI 3 (2016): 54.
[2] Chaouch, Naira, et al. “A synergetic use of satellite imagery from SAR and optical sensors to improve coastal flood mapping in the Gulf of Mexico.” Hydrological Processes 26.11 (2012): 1617-1628.
[3] Klemas, Victor. “Remote sensing of floods and flood-prone areas: an overview.” Journal of Coastal Research 31.4 (2014): 1005-1013.
[4] Ogashawara, Igor, Marcelo Pedroso Curtarelli, and Celso M. Ferreira. “The use of optical remote sensing for mapping flooded areas.” International Journal of Engineering Research and Application 3.5 (2013): 1-5.
[5] Ticehurst, C. J., P. Dyce, and J. P. Guerschman. “Using passive microwave and optical remote sensing to monitor flood inundation in support of hydrologic modelling.” Interfacing modelling and simulation with mathematical and computational sciences, 18th World IMACS/MODSIM Congress. 2009.

We recommend also having a look at the last year’s task papers in the MediaEval 2017 Proceedings:

Guillaume Gravier et al. (eds.) 2017. Proceedings of the MediaEval 2017 Workshop, Dublin, Ireland, Sept. 13-15, 2017.

Task organizers
Benjamin Bischke, German Research Center for Artificial Intelligence (DFKI), Germany benjamin.bischke @
Patrick Helber, German Research Center for Artificial Intelligence (DFKI), Germany
Zhengyu Zhao, Radboud University, Netherlands
Jens de Bruijn, Floodtags, Netherlands
Damian Borth, German Research Center for Artificial Intelligence (DFKI), Germany

Task schedule
Development data release: 31 May 2018
Test data release: 30 June 2018
Runs due: 30 September 2018
Working Notes paper due: 17 October 2018