By Fusing Images, Lehigh Professor Detects Concealed Weapons

As VIPs line up for a reception with the President and foreign leaders, impassive security guards look each person up and down, alert to any clue of malicious intent.

Rick Blum, professor of electrical and computer engineering, hopes to equip these soldiers and security guards with a device that can save them a few potentially life-saving seconds in the search for concealed weapons.

Blum and his graduate students have devised a system that combines a photo taken by an optical camera with a photo of the same subject taken by a millimeter-wave camera (MMW). The result is a composite photo that exposes much more than either photo reveals by itself.

Blum's web page (http://www.ece.lehigh.edu/SPCRL/spcrl.htm) illustrates this application of "image fusion" with a striking set of three photos, arranged side by side, each showing the same shot of the same three men. In the left image, taken with an optical camera, you can see the men's faces and clothing. In the center image, taken with an MMW camera you see no physical features or clothing, but you can peer past their clothing and see that the man on the right has a gun underneath his sweater.

On the right is the fused image, which police and military personnel are seeking. Here, you see enough of the men's clothing and features to tell them apart. You can also make out the unmistakable outline of a gun under the third man's sweater.

Image fusion has found other applications. Using only a digital camera, for example, Blum and his students have fused two photos of the same setting so that an alarm clock in the foreground and a man seated in the background are both in focus.

Blum's research group has developed new image-fusion algorithms that perform better than all existing approaches. They are one of the few groups to perform careful performance analysis and classification of existing approaches. Recently, Blum and his students have begun preliminary studies of a portable image-fusion system using wireless communication links.

Blum, whose research specialties are wireless communications and signal processing, originally had no intention of making a foray into concealed-weapon detection. Eight years ago, however, while doing research at the U.S. Air Force Research Laboratory in Rome, N.Y., he met researchers from the Georgia Institute of Technology who were studying sensors for a law-enforcement client. Blum quickly saw a natural fit between signal processing and sensors. He also met police, who described their need for a single image that would reveal both weapon and scene, thus enabling them to best use their knowledge and experience to quickly assess a situation.

Blum's research has been supported by the National Science Foundation, the State of Pennsylvania, AT&T, the U.S. Army Research Office, the Office of Naval Research and the Air Force Office of Scientific Research. He is seeking additional funding to construct a portable image-fusion system using wireless communication links.

With a portable system, the guards on duty at the presidential VIP reception could scan the fused and rapidly transmitted images of arriving guests unobserved, from a hidden room and in real time.

Another advantage of Blum's system is that, unlike x-rays, MMW cause no physical harm to a person and can thus be taken without a person's permission or knowledge.

One obstacle Blum has encountered is cost. MMW cameras are far too expensive for the typical police department budget; in fact, there are only a handful of the cameras in the U.S. Advances in sensor are very likely to overcome this problem. As an alternate approach, Blum has investigated using infrared sensors in place of MMW cameras. Thus far the deficiencies of the infrared sensors have hindered progress but his efforts continue.

In Blum's algorithms, image fusion is accomplished in the wavelet transform domain. The two images to be fused are first processed with a wavelet transform. Fusion is then accomplished by combining the wavelet transforms, and the fused image is obtained by applying an inverse wavelet transform. Blum's research group was the first to provide a very general description of the class of algorithms of this type. Fusing the images in the wavelet transform domain avoids the problem of incompatible pixels, which can result when images are fused in pixel domain.

Blum sees a growing need for concealed weapons detection in Iraq, where more than 40 American soldiers have been killed, many at checkpoints, since major hostilities ended in April.

"It would really be great to be able to give this technology to police and to military troops in Iraq," Blum says, "so they can determine as quickly as possible whether or not there is an issue."

Blum is co-writing a book called Multi-Sensor Image Fusion and Its Applications for Marcel Dekker publishers.