Carnegie Mellon Professor's Unique New Vision Technology Will Be Used To Present Replays In Super Bowl XXXV

The action at Super Bowl XXXV will be captured by more than 30 cameras, each poised some 80 feet above the field at Raymond James Stadium in Tampa, Fla. Each camera, with computer-controlled zoom and focus capabilities, is mounted on a custom-built, robotic pan-tilt head, which can swing the camera in any direction at the command of a computer. These camera heads are controlled in concert so that cameras point, zoom and focus at the same time on the same spot on the field, where some action --touch down or fumble--is occurring.

The system operates in the following way: One of the camera heads is designated as the master camera. A human cameraman operates a movable pan-tilt tripod, attached to a TV screen on which the video from the master camera is constantly displayed. The pan-tilt tripod is equipped with sensors to constantly measure its angle. The master camera head moves by mimicking the motion of the tripod as the cameraman moves it to capture a moving object on the field on his TV screen. In the meantime, information collected from the master camera's pan-tilt angles, zoom and focus is fed to a computer, which quickly computes the appropriate control signal for each of the remaining cameras. This causes all of them to converge on the same target and capture its image from a variety of angles.

Live action on the football field is continuously captured up to 30 times per second by the video cameras. The video of each camera is synchronized and time stamped so that all the views at the most critical and interesting moments can be played back in sequence, as if a viewer had flown around the action.

Kanade will explain his technology in an interview from Tampa, which will air during the Super Bowl Pre-Game Show. He notes that the "Eye Vision" demonstration that will appear on Super Bowl Sunday is only a small part of this new technology, which he calls "Virtualized Reality," as opposed to virtual reality, and is the product of more than six years of research.

For Virtualized Reality to achieve its full impact, the set of captured, multiple video images must be processed beyond the play back. The detailed geometrical information about a scene- - the shapes of targets and background--is extracted by computer, which enables a person to choose how to view a scene, even from a perspective that was actually not shot by any camera.

To bring this concept to life, Kanade and his students built a "3D room" equipped with more than 50 video cameras and experimented by filming people involved in a variety of sports activities. He also spun off a company named Zaxel Systems, Inc., for commercialization of the technology. Much of this work can be viewed at the Virtualized Reality Web site: http://www.cs.cmu.edu/afs/cs/project/VirtualizedR/www/VirtualizedR.html.

In contrast to virtual reality, in which synthetic environments are created, Virtualized Reality, and to a lesser extent, Eye Vision, are based on events taking place in the real world, which are captured and processed by computer manipulation.

"Because our models are derived from real images," Kanade says, "the models look much more real than typical virtual worlds."

Kanade says the output from these multiple cameras shooting a scene together from many angles actually can create totally new views that were not captured by any camera. As this technology develops, he believes it will create a completely new way to view sports and entertainment events. People will be able to customize the perspective from which they watch – e.g. from that of a particular player or the ball.

Kanade is the director of Carnegie Mellon University's Robotics Institute. He has been a leader in the development of video-based vision systems used in the university's autonomous vehicles and exploration robots. His team has developed a vision-based autonomous helicopter, which ultimately may be able to aid in search and rescue operations, fire fighting and inspection tasks. He is also a pioneer in medical robotics and computer-assisted surgery, working with surgeons and medical professionals to develop smart tools capable of performing medical procedures better than a physician or machine could do alone.

Kanade earned his undergraduate and graduate degrees in electrical engineering from Kyoto University, Japan. He has been on the Carnegie Mellon faculty since 1980 and director of the Robotics Institute since 1991. He is a member of the National Academy of Engineering. Most recently he received a $100,000 award from the NEC Foundation for C&C Promotion for "fundamental and broad contributions to the development of multimedia through the advancement of robotics and computer vision."