Multimedia search without detours

Fingerprints at the scene of a crime can betray the perpetrator. And digital fingerprints can help to locate video and audio files -- a party video on the Internet, an advertising spot in a television program, or the original version of a film extract in the broadcaster's archive. Fingerprints in music files can provide information on its tempo, genre or the proportion of rhythmic instruments. Those used in videos may contain information on scene changes, camera movements and the brightness of the image. In contrast to manually captured metadata, automatically generated fingerprints are never ambiguous. In many archives -- for example those of television broadcasters -- they are generated when the file is initially stored. This differs from the Internet, where fingerprints have to be recreated with every search. For this purpose, it was previously necessary to decompress compressed files. But that is no longer the case with the multimedia search engine DIVAS, short for direct video & audio content search engine. DIVAS finds the file you are looking for without needing to first unpack the entire media archive.

The search engine is suitable not only for searches on the Internet and in archives, but also for monitoring TV programs, for example to check whether a contractually agreed advertisement has been broadcast. As the software can take "fingerprints" of compressed files, it is faster than comparable search engines. "Our series of tests with MP3 files showed that search times can be halved," reports Prof. Gerald Schuller of the Fraunhofer Institute for Digital Media Technology IDMT in Ilmenau. Together with Matthias Gruhne he was responsible for developing the audio component of DIVAS, which covers the extraction of fingerprints and the subsequent search. Both of these function even with music files that are recorded from a loudspeaker using a cell phone. "Even if the sound has been severely distorted, our methods make it possible to clearly identify and classify the song," explains Schuller.

The fingerprints are stored in MPEG-7 format, the ISO standard for multimedia data. DIVAS can be used to detect similarities between different video or audio contents as well as for topic-related data searches, for instance to find songs of a certain genre. The software can handle audio and video formats such as MP3, AAC, H.264 and MPEG-2. DIVAS was the brainchild of an EU-wide joint project in which seven partners from seven different countries took part. They included Belgian television broadcaster BETV, which plans to use the multimedia search engine in its own archive.