More realistic computer graphics

The new techniques focus on "real time" graphics which need to maintain the illusion of interactivity as scenes change in response to user moves. These graphics can be used in applications such as video games, extended reality, and scientific visualization tools.

Both papers demonstrate how developers can create sophisticated lighting effects by adapting a popular rendering technique known as ray tracing.

"Over the last decade, ray tracing has dramatically increased the realism and visual richness of computer-generated images in movies where producing just a single frame can take hours," said Wojciech Jarosz, an associate professor of computer science at Dartmouth who served as the senior researcher for both projects. "Our papers describe two very different approaches for bringing realistic ray-traced lighting to the constraints of real time graphics."

The first project, developed with NVIDIA, envisions the possibilities for future games once developers incorporate NVIDIA's hardware-accelerated RTX ray tracing platform. Recent games have started to use RTX for physically correct shadows and reflections, but quality and complexity of lighting is currently limited by the small number of rays that can be traced per frame.

The new technique, called reservoir-based spatiotemporal importance resampling (ReSTIR), creates realistic lighting and shadows from millions of artificial light sources. The ReSTIR approach dramatically increases the quality of rendering on a computer's graphics card by reusing rays that were traced in neighboring pixels and in prior frames.

The new technique can be integrated into the design of future games and works up to 65 times faster than previous rendering techniques.

"This technology is not just exciting for what it can bring to real-time applications like games, but also its impact in the movie industry and beyond," said Benedikt Bitterli, a PhD student at Dartmouth who served as the first author of a research paper on the technique.

The second project, conducted in collaboration with Activision, describes how the video game publisher has incorporated increasingly realistic lighting effects into its games.

Traditionally, video games create lighting sequences in real time using what are called "baked" solutions: the complex ray-traced illumination is computed only once through a time-consuming process. The lighting created using this technique can be displayed easily during gameplay, but it is constrained to assuming a fixed configuration for a scene. As a result, the lighting cannot easily react to the movement of characters and cameras.

The research paper describes how Activision gradually evolved its "UberBake" system from the static approach to one which can depict subtle lighting changes in response to player interactions, such as turning lights on and off, or opening and closing doors.

Since UberBake was developed over many years to work on current games, it needed to work on a variety of existing hardware, ranging from high-end PCs to previous-generation gaming consoles.

"Video games are used by millions of people around the world," said Dario Seyb, a PhD student at Dartmouth who served as the research paper's co-first author. "With so many people interacting with video games, this technology can have a huge impact."

Dartmouth researchers on both projects are affiliated with the Dartmouth Visual Computing Lab.

"These industry collaborations have been fantastic. They allow our students to work on foundational academic research informed by practical problems in industry, allowing the work to have a more immediate, real-world impact," said Jarosz.

The research papers will be published in ACM Transactions on Graphics and presented at SIGGRAPH 2020 taking place online during the summer.