The internet of meetings

The future of corporate meetings and college lectures will be interactive, multimedia, searchable and cross-linked. It will, finally, unlock the unrealised power of meetings to focus efforts, pool resources and stimulate imagination to discover creative answers to obdurate questions.

That future is just about here.

All thanks to AMI, the Augmented Multiparty Interaction consortium, a large, well-resourced consortium that has devoted two projects, dozens of partners and millions of euros to apply the latest technology to one of the oldest corporate exercises.

The first project was AMI, which developed a browser for all captured meeting information in audio, video and text, while the second project, AMIDA, enhanced the browser technologies and added distance access.

Beyond the boardroom

The potential of these technologies extend far beyond the boardroom and is currently being sold to universities,government institutions and businesses including the Open University, UNICEF and Nestle. AMI also talked with the Davos Forum organisers about deploying the technology for its annual meeting of the world's business leaders.

It all began with an attempt to make meetings more efficient, effective and much more powerful, by capturing all the information, ideas and exchanges as they happen. Audio, video and text capture technologies using signal processing, artificial intelligence (AI), machine learning, and social dynamics.

"We did not develop any hardware, rather we developed software and adapted it to off-the-shelf components," says Hervé Bourlard, director of the IDIAP Research Institute, Professor at the Swiss Federal Institute of Technology at Lausanne and coordinator of the AMIDA project (http://www.amiproject.org/), together with Prof. Steve Renals of the University of Edinburgh. The system can even capture notes taken by meeting participants using a Logitech pen.

Transcription, fast and accurate

Audio capture, too, experienced significant advances on the state of the art. First developed in AMI and perfected in AMIDA, it can transcribe text in real time, with high accuracy in meetings.

"That's more than enough for a meeting," explains Bourlard. "And the transcription software is particularly adept at picking up keywords. In any case, the transcript is annotated with the recording, so if you want to hear exactly what was said and how it was said at that point in the transcript, you can simply click a button and hear the original instantly."

It even works with web-based audio, using an adapted application called WebASR. That application is very accurate, which makes it exploitable in many forms and it has proved hugely popular, both among other researchers and the general public.

Transcription software is very expensive and commercial applications only achieve 90 percent accuracy after some training. But AMIDA is approaching that level of accuracy with a real-time application that requires no training and one which must deal with a vast number of accents.

The consortium is currently exploring the potential of commercialising technology from this work.

An internet of meetings

The driving idea behind these developments is to massively enhance the value and effectiveness of meetings, and to capture much of the lost, intangible information that never reaches a traditional meeting's minutes.

In addition to transcription and annotation in real time, the software can summarise the content based on a number of criteria, and link all this information to relevant in-meeting documents and presentations.

Taken together, the AMI and AMIDA tools represent a sort of 'internet of meetings', an application of the easy knowledge discovery enabled by the world wide web, but applied to a very special context, safe behind the corporate firewall, accessible only by authorised staff or authorised visitors.

In one amazing advance, a new software module can offer background documents from the corporations archives, based on the content of the conversation as it happens. When the software hears keywords it can search the archives for relevant supporting material.

All of this work can extend beyond meetings and lectures. The keyword recognition could be adapted to improve the way commands are given to robots.

Content linking is another strength of the platform. Presentations and documents used in the meeting are automatically captured, and the software can even track items written on the whiteboard using a form of radar. Now when a whiteboard is wiped clean, nothing is lost.

In the context of meetings, the power of this technology is immediately apparent. If there is conflict between participants over what they believe was decided, AMI can settle the dispute. This is just a simple, value-added benefit of the AMI system.

Finding lost human capital

But arguably the greatest benefit to corporations comes over time, as an archive of meetings develops. Meetings dealing with different problems, different types of project, different clients with related issues, or the same client with different issues, can be accessed by search, and reviewed rapidly using meeting summaries.

The upshot is that, when an issue arises, the solution or best approach may already exist in the corporation's meeting archives. This human capital is normally lost to an enterprise, or restricted to the individual or individuals concerned, who may work in another department, branch or country. But with AMI that human capital, experience, imagination, insight, unrepeatable brainwaves are captured on the fly, in real time, for all time.

The AMI consortium currently has access to four state-of-the-art smart meeting rooms, with multiple cameras, audio inputs, data capture-enabled whiteboards and other high-tech equipment.

"Right now, that technology occupies a series of racks, but in the near future we will be able to reduce that hardware to a single notebook," explains Bourlard. There are obvious commercialisation routes to such a compact system.

Commercial success

The core technologies are now a major commercial success and the source of a spin-off company, called Klewel, competing internationally. Klewel's software suite is called the Lecture Acquisition System, or LAS.

Klewel provides multimedia support to conferences, offering audio, video and presentation capture, all unobtrusively. Speakers do not need to provide original slides, a major headache in many conferences.

The data is then uploaded to Klewel's secure servers, where knowledge processing takes place. This solution automatically references the full content of presentations, including audio, video and slides.

The content of the slides is automatically indexed thanks to Klewel's AMI-derived technology. Multimedia files are encoded into a format suitable for web video streaming. These presentations are quickly published and then fully accessible from an intranet or website.

Rich e-learning

Interested parties can immediately access and retrieve specific information without having the need to play back the full presentation. Search using keywords will retrieve relevant information from all archived presentations.

And keyword search works across multiple conferences. Audio, video and slides are also synchronised. Users can click on the related slides and watch and listen to the talk from that slide onward, and Klewel's player can easily be integrated into any webpage.

Imagine such a system used for college lectures? This is what is provided, in a much more primitive form, by MIT's opencourseware. Klewel's technology adds search and annotation. Over time, this work can acquire vast knowledge and offer an extremely rich and flexible e-learning opportunity.

And Bourlard believes this is just the beginning. AMI's work so far has created commercial successes, with more to come, but the greatest advances in augmented multiparty interaction remain to be discovered. The work of the AMI and AMIDA projects has created a platform that makes those discoveries possible.

The AMIDA project received funding from the ICT strand of the EU's Sixth Framework Programme for research.