Task Results
- Task overview: [slides][video]
- Task results: [presentation videos]
- For all papers, see the MediaEval 2018 Working Notes Proceedings: http://ceur-ws.org/Vol-2283
Task description
In this task, we ask participants to investigate ways to summarize how e-sport matches ramp up, evolve and play out over time. The ultimate goal is to generate an engaging and captivating story, which boils down the thrill of the game to its mere essence, the GameStory (see illustration below). As a pilot task at MediaEval, we welcome participants to be creative, think out of the box, and communicate and exchange their approaches, even if they tackle only part of the problem. In this regard, we will offer match-making of interested parties if
they want to participate but conceive of their team as too small.
Training and test data for the task are provided in cooperation with ZNIPE.TV, which is a rapidly growing platform for e-sport streaming. Participants are asked to create a specific number of summaries, which are evaluated by an expert panel. The expert panel will include professionals from ZNIPE.TV and researchers from the field of game studies and narratives in video games. The exact criteria for evaluating submissions will be available to the participants within the in-depth task description.
Task motivation
E-sports is huge. Already in 2013 concurrent users for a single event exceeded eight million for a League of Legends Championship. In 2017 approximately 143 million viewers accessed e-sports streams frequently. The rich bouquet of data including audio and video streams, commentaries, game data and statistics, interaction traces, viewer-to-viewer communication, and many more channels allow for particularly challenging multimedia research questions.
One of the key challenges of the industry today is to render the results of e-sports events more promotable. It is notoriously difficult to search, e.g., for exciting highlights, because of the huge amounts of video recorded for each match and their homogeneity: for example, every match of League of Legends roughly looks the same. A lot depends on the manual selection and curation of supplementary material for promotion on an event’s website. This way, individual matches of e-sports events can be more easily accessed by those attending on-site or by stream, however, at the cost of significant manual overhead. At the same time, these efforts also enable the event and its highlights to be retrieved or recommended later on.
Target group
The target groups of researchers for this task is diverse and broad. It includes researchers from multimedia, web science, data science, machine learning, information retrieval, natural language processing, and game and entertainment-related fields. We also encourage interdisciplinary research involving people from the media sciences, the cultural sciences, and the humanities discussing what would be part of such a story and what the expectations of the audience are. In any case, regardless the research background, it will help to have a basic understanding of the gaming culture, or ideally, to enlist dedicated gamers.
Data
Training and test data for the task are provided in cooperation with ZNIPE.TV. There are more than 500 matches of Counter-Strike: Global Offensive , short CS:GO , available for the task featuring matches from the Electronic Sports League (ESL), Intel Extreme Masters (IEM), Dreamhack, and more. Experts from ZNIPE.TV will select 10 matches for development and 3 matches as test data set for this first year of the task. All of the matches have video streams from the 10 gamers (five per team), a spectator video stream with commentator audio, a video stream with the map overview, structured data on game statistics and game highlights, and interaction patterns from viewers using ZNIPE.TV. We will provide visual and audio features for the video streams as well as transcripts generated automatically from the commentator stream.
Ground truth and evaluation
As the GameStory task encourages an explorative approach, we will not provide a ground truth in the classical sense. We will provide links to summaries of E-sport matches that give an impression on what the jury appreciates. They may be mixed from fans and from the ZNIPE.TV content team (which for sure also are fans) and they may be more or less popular. To foster novel approaches, including experiments with visualization and audio streams, we ask the jury explicitly to point out innovative and surprisingly new approaches to creating a story summarizing the matches. Participants are asked to submit up to two stories per match from the test data set. There is no constraint on the modalities of the story, so it can be video, audio, text, images or a combination thereof. An expert panel with professionals from ZNIPE.TV and researchers from the field of game studies and non-linear narratives will then investigate the submissions and judge them for:
● Conciseness and accurateness (ie. does the story recite the match?)
● Is it exciting, compelling, engaging, does it provide flow and peak of a good story?
● Innovation (ie. surprisingly new approach, non-linearity of the story, creative use of cuts, etc.)
● Simplicity and degree of automation of the approach
● Cross-domain applicability
Recommended reading
[1] J. Bryce & J. Rutter, 2002. "Spectacle of the Deathmatch: Character and Narrative in First Person Shooters" in G. King & T. Krzywinska (Ed.s), ScreenPlay: Cinema/videogames/interfaces, Wallflower Press, pp.66-80
[2] Shah, Rajiv Ratn, et al. "Eventbuilder: Real-time multimedia event summarization by visualizing social media." Proceedings of the 23rd ACM international conference on Multimedia. ACM, 2015.
[3] Lee, Wen-Yu, Winston H. Hsu, and Shin’ichi Satoh. "Learning From Cross-Domain Media Streams for Event-of-Interest Discovery." IEEE Transactions on Multimedia 20.1 (2018): 142-154.
[4] M. Seif El-Nasr, A. Drachen, A. Canossa (Eds.). “Game Analytics”. Springer, 2013.
Task organizers
Duc-Tien Dang-Nguyen, duc-tien.dang-nguyen at dcu.ie, Dublin City University
Marcus Larson, ZNIPE.TV
Martin Potthast, Universität Leipzig
Mathias Lux, mlux at itec.aau.at, Alpen-Adria-Universität Klagenfurt
Michael Riegler, SIMETRIC & University of Oslo, michael at simula.no
Pål Halvorsen, Simula Research Laboratory & University of Oslo
Task schedule
Development data release: 30 May 2018
Test data release: 2 July 2018
Runs due: 17 September 2018 (updated deadline)
Results returned: 24 September 2018 (updated deadline)
Working Notes paper due: 30 September 2018