Projection mapping leaves the darkness behind

Conventional projection mapping, which turns any three-dimensional surface into an interactive display, requires darkness because any illumination in the surroundings also illuminates the surface of the target object used for display. This means that black and dark colors appear too bright and can't be displayed properly. In addition, the projected images always look like they are glowing, but not all real objects are luminous, which restricts the range of objects that can be displayed. Displays in dark environments have another disadvantage. Multiple viewers can interact with an illuminated scene, but they are less able to interact with each other in the dark environment.

"To get around this problem, we use projectors to reproduce normal illumination on every part of the room except the display object itself," says Masaki Takeuchi, lead author of the study. "In essence, we create the illusion of global illumination without using actual global illumination."

Projecting global illumination requires a set of techniques that differs from those of conventional projection mapping. The research team use several standard projectors to illuminate the room along with a projector with a wide aperture and large-format lens to soften the crisp edges of shadows. These luminaire projectors illuminate the environment, but the target object remains in shadow. Conventional texture projectors are then used to map the texture onto its shadowed surface.

The researchers built a prototype environment and evaluated the performance of their approach. One aspect they evaluated was whether the objects were perceived by humans in aperture-color mode (where the colors appear to radiate from the object itself) or surface-color mode (in which the light appears to be reflected from a colored surface).

"To our knowledge, we are the first to consider this," says Daisuke Iwai, senior author of the study. "However, we believe it is fundamental for producing realistic environments."

The researchers found that, using their method, they could project texture images onto objects without making the object appear to glow. Instead, the textures were perceived to be the true colors of the object's surface.

In future, the researchers plan to add more projectors to handle the complex illumination in the areas next to the display object. Eventually, they aim to produce scenes that are indistinguishable from real-world three-dimensional scenes. They believe that this approach will enable visual design environments for industrial products or packaging, in which the participants can interact not only with their design under natural light but also with each other, facilitating communication and improving design performance.