Computer Graphics Researchers Simulate The Sounds Of Water And Other Liquids

Those are some of the sounds that have been missing from computer graphic simulations of water and other fluids, according to researchers in Cornell's Department of Computer Science, who have come up with new algorithms to simulate such sounds to go with the images.

The work by Doug James, associate professor of computer science, and graduate student Changxi Zheng will be reported at the 2009 ACM SIGGRAPH conference Aug. 3-7 in New Orleans. It is the first step in a broader research program on sound synthesis supported by a $1.2 million grant from the Human Centered Computing Program of the National Science Foundation (NSF) to James, assistant professor Kavita Bala and associate professor Steve Marschner.

In computer-animated movies, sound can be added after the fact from recordings or by Foley artists. But as virtual worlds grow increasingly interactive and immersive, the researchers point out, sounds will need to be generated automatically to fit events that can't be predicted in advance. Recordings can be cued in, but can be repetitive and not always well matched to what's happening.

"We have no way to efficiently compute the sounds of water splashing, paper crumpling, hands clapping, wind in trees or a wine glass dropped onto the floor," the researchers said in their research proposal.

Along with fluid sounds, the research also will simulate sounds made by objects in contact, like a bin of Legos; the noisy vibrations of thin shells, like trash cans or cymbals; and the sounds of brittle fracture, like breaking glass and the clattering of the resulting debris.

All the simulations will be based on the physics of the objects being simulated in computer graphics, calculating how those objects would vibrate if they actually existed, and how those vibrations would produce acoustic waves in the air. Physics-based simulations also can be used in design, just as visual simulation is now, James said. "You can tell what it's going to sound like before you build it," he explained, noting that a lot of effort often goes into making things quieter.

In their SIGGRAPH paper, Zheng and James report that most of the sounds of water are created by tiny air bubbles that form as water pours and splashes. Moving water traps air bubbles on the scale of a millimeter or so. Surface tension contracts the bubbles, compressing the air inside until it pushes back and expands the bubble. The repeated expansion and contraction over milliseconds generates vibrations in the water that eventually make its surface vibrate, acting like a loudspeaker to create sound waves in the air.

The simulation method developed by the Cornell researchers starts with the geometry of the scene, figures out where the bubbles would be and how they're moving, computes the expected vibrations and finally the sounds they would produce. The simulation is done on a highly parallel computer, with each processor computing the effects of multiple bubbles. The researchers have fine-tuned the results by comparing their simulations with real water sounds.

Demonstration videos of simulations of falling, pouring, splashing and babbling water are available at http://www.cs.cornell.edu/projects/HarmonicFluids.

The current methods still require hours of offline computing time, and work best on compact sound sources, the researchers noted, but they said further development should make possible the real-time performance needed for interactive virtual environments and deal with larger sound sources such as swimming pools or perhaps even Niagara Falls. They also plan to approach the more complex collections of bubbles in foam or plumes.

The research reported in the SIGGRAPH paper was supported in part by an NSF Faculty Early Career Award to James, and by the Alfred P. Sloan Foundation, Pixar, Intel and Autodesk.