Enabling video systems to react intelligently to content

The EUREKA ITEA 2 software Cluster CANTATA project has made possible advanced and automatic analysis of digital video content to improve surveillance, speed medical diagnosis in life-threatening situations and provide consumers with fast access to preferred home entertainment material. Advanced digital technologies and the availability of greater systems power have enabled the development of robust analytical algorithms for content interpretation, a scalable platform facilitating analysis across a wide range of applications, content presentation that adapts to the device, user and content, and a common understanding of quality levels in content analysis. Demonstrations of surveillance, medical and multimedia applications developed for the project are already leading to commercial products.

The amount of digital video content now available makes automated interpretation essential to ensure optimal use of data in the shortest possible time. Massive expansion of video applications over the Internet and corporate networks is becoming a reality and additional content-analysis-based applications are about to follow.

Such applications involve transferring and interpreting huge amounts of data -- from multi-camera security installations functioning 24 hours a day, 7 days a week, through the increasing amount of graphical information in medical diagnosis from for example high resolution 3D scanners that put a high stress on the medical experts involved, to the homeconsumer who has access to virtually unlimited multimedia content, increasingly over the Internet.

Making the system aware

Current video systems are limited as human interpretation is required to discriminate. The ITEA CANTATA project saw a need to make the system itself aware of both the contents and its context, and to use this knowledge to establish an action or control the environment autonomously. As a result, the quality and reliability of automatic content analysis will become an important discriminating factor in market choice.

"Even intelligent state-of-the-art systems currently fail compared with human intelligence as the context of information on which the adaptive behaviour is based is completely ignored," explains Dr. Egbert Jaspers from Philips Research. "Realistic context models are needed to improve the decision making of complex vision systems and to produce faithful and meaningful behaviour."

CANTATA set out to bridge the gap between academic research into such content analysis and economic feasibility. "Bringing together universities, large companies and small and medium-sized enterprises (SMEs) enabled a wide range of competencies to be focused on the challenges," says Jaspers. "Such co-operation is of great benefit to SMEs as they can't normally afford such a level of investment in research. Moreover, the project was particular suited to the EUREKA ITEA Cluster."

Developments included: algorithms for content analysis in different domains; an analytical and presentation platform suitable for all areas; visualisation and user interaction, focusing mainly on home consumers; and methodology for validation of content-aware products.

Fast commercial exploitation possible

Key innovations were demonstrated in home multimedia, video surveillance and medical diagnostics and the results have been exploited commercially very quickly after the end of the project:

Home multimedia -- CANTATA developed a content-aware interactive TV system that offers a host of new features deploying awareness of the multimedia content. For example, it can automatically summarise broadcast news and sports items, showing only highlights such as goals. And it can recommend content based on user mood and preferences as well as on multimedia content played before. These concepts are already being incorporated in home-entertainment products, such as a content-aware Internet TV set-top box from Ortikon.

Video surveillance -- CANTATA developed an intelligent surveillance camera combining advanced video-content analysis and state-of-the-art video compression for streaming over Internet. Robust software algorithms are able to recognise automatically the posture of human beings, allowing detection of abnormal behaviour. The results have already been incorporated into VDG Security video-surveillance systems offering multiple camera and multiple video-content analysis channels. Applications include a 300-camera system covering all 32 stations in the Charleroi metro system in Belgium with central control and fast reaction to incidents. And a 120-camera system was installed for the 2009 Dubai Formula One race.

Healthcare -- CANTATA developed automated diagnosis of deep-vein thrombosis and pulmonary embolisms. A pulmonary embolism is difficult to detect yet timely diagnosis and appropriate therapy can reduce mortality to under 10%. While multi-detector computed tomography (MDCT) has radically improved diagnosis, it requires radiologists to check hundreds of images -- a time-consuming process subject to human failure. The CANTATA system not only detects embolisms automatically and reliably but also provides a compressed image for transmission over bandwidth-limited networks without affecting the quality of the medical content. Clinical evaluation was carried out in several Dutch hospitals. The first computer-aided detection systems for pulmonary embolisms from Philips Healthcare are already entering the market.

Boosting company growth

In addition, the ITEA project enabled many of the partners involved to grow. Spin-off companies have developed significantly since the start of the project. For example, high-tech start-up ViNotion, which specialises in automation systems based on video-content analysis, has grown from zero to six full-time equivalents. And VDG Security has nearly tripled its research staff to 20.