Data-driven music: Converting climate measurements into music

"I strongly hope that this manuscript marks a significant turning point, transitioning from an era where only scientists handle data to one where artists can freely leverage data to craft their works," writes author and composer Hiroto Nagai, a geo-environmental scientist at Rissho University.

Scientist-composer Hiroto Nagai asserts that music, as opposed to sound, evokes an emotional response and posits that "musification" (as opposed to sonification) of data requires some intervention by the composer to build tension and add dynamics. For this reason, Nagai was more liberal in adding a "human touch" compared to previous data-based musical compositions, aiming to meld sonification with traditional music composition.

"As a fundamental principle in music composition, it is necessary to combine temporal sequences from tension-building to resolution in various scales, from harmonic progressions to entire movements," Nagai writes. "So far, there haven't been published attempts and open discussion on sonification-based music composition, nor attempts to demonstrate the methodology required to intentionally affect the audience's emotions with an artistic piece."

To do this, he first used a program to sonify environmental data by assigning sounds to different data values. The publicly available data was collected from four polar locations between 1982 and 2022: an ice-core drilling site in the Greenland ice sheet, a satellite station in Norway's Svalbard archipelago, and two Japanese-owned research stations in the Antarctic (Showa Station and Dome Fuji Station). For each of the sites, Nagai used data on monthly measurements of short- and longwave radiation, precipitation, surface temperature, and cloud thickness.

In the next step, he transformed this collection of sounds into a musical composition to be played by two violins, a viola, and a cello. This process involved many steps, including manipulating the pitch of different datapoints and assigning sections of data to the different instruments, overlaying passages created from different data, and introducing musical playing techniques such as pizzicato and staccato. Nagai also intervened in more artistic ways by introducing rhythm, deliberately removing certain sounds, and introducing handwritten (non-data derived) parts into the composition.

The quartet's premiere live performance was shared at Waseda University in Tokyo in March 2023 followed by a panel discussion. A filmed performance of the piece by PRT Quartet, a Japanese professional string quartet, was also released on YouTube in March 2023.

"Upon listening, my initial reaction was like, 'What is this?' It felt like a typical contemporary piece," said Haruka Sakuma, the professional violinist who performed 2^nd violin. "The flow of the music was a bit hard to memorize quickly, and it was quite challenging at first."

Nagai says that, in contrast to graphical representations of data, music elicits emotion before intellectual curiosity and suggests that using graphical and music representations of data in conjunction might be even more powerful.

"It grabs the audiences' attention forcefully, while graphical representations require active and conscious recognition instead," Nagai writes. "This reveals the potential for outreach in the Earth sciences through music."

This research was supported by the Remote Sensing Technology Center of Japan.