Welcome to the portal with supporting material for MuseSync!
MuseSync was a response to the 2012 ACM Multimedia Grand Challenge on Automatic Music Video Generation. In this, the challenge was posed to auto-suggest cool soundtracks for user-generated web videos.
In our response to the challenge, we built a system which uses automated multimedia content analysis techniques inspired by findings from music psychology and musicology, and give an important role to the intended narrative of the video. We connect the narrative to best practices in professional movies by using knowledge from collaborative web resources.
The full MuseSync pipeline could be visualized as follows:
For the further background of the MuseSync system, please consult our extended abstract:
Our supplementary material is split up into several parts:
- Full versions of the synchronized videos that were shown at the demo presentation are on YouTube: first example (hideout investigation, drums), second example (hideout investigation, hip-hop) and third example (adventurer, dungeon).
- Want to see more examples? We also put up a playlist on YouTube with more examples of song suggestions and synchronizations for different videos.
- For those interested in setting up the system themselves, please consult the MuseSync deployment guide first...
- ...and then proceed by performing an SVN checkout to obtain the MuseSync source code. The repository URL is https://svn-mede.ewi.tudelft.nl/insy-msp-cynthialiem/musesync/. Due to access restriction policies, you will need to provide credentials in order to reach the repository: please use username anonymous and password anonymous for this.
- Upon checking out the code, you will need to recreate the music repository. For convenience, we provide a tarball with all repository music files here (WARNING: large file, over 6 GB!). We kindly request you not to rehost this data, and respect the attribution wishes of the individual composers as stated on their original websites at incompetech.com, danosongs.com and derekaudette.ottawaarts.com. The MP3 files should be unpacked or copied to the currently empty folder that will be found in the source tree under matlab/musictracks.
Finally, you may also want some test videos. You can record and use your own (see the deployment guide), but a colleague of ours kindly granted us permission to reuse, host and experiment with a collection of cat videos (WARNING: large file, 1.5 GB) he has been recording over the last year (thanks Raynor!). Enjoy!