The piano as a typewriter

To develop a mapping from language to music, the researchers analyzed hundreds of music pieces to find frequent motor patterns. "We had to respect the note transitions and chords that occur frequently in music. No pianist can quickly play dissonant chords or very large intervals, thus our mapping had to avoid these," said Anna Feit, researcher at Max Planck Institute for Informatics.

The mapping was optimized for the English language. Therefore the researchers checked the distributions of letters and letter sequences in English texts. Then they developed a computational approach that allows addressing the enormous number of possible mappings. There are more than 10⁴⁸ possibilities in mapping the 26 letters of the English alphabet to the 88 keys on the piano keyboard.

"Our approach ensures that frequent letter sequences are translated into melodic structures that are well known and can therefore be played quickly by a pianist," said Feit. In this way, frequent letter pairs like "th" or "he" were translated to a third or a fifth -- intervals that are very well practiced by every pianist. The letter "e," which occurs most frequently in English, was mapped to different notes in different octaves. Furthermore, frequent syllables and words were mapped to chords of the major and minor scales.

To assess whether the resulting keyboard was good enough for typing, the computer scientists first conducted a study with a piano professor. He was asked to "play sentences" that were translated into music pieces shown on a sheet. "Without prior practice he was able to enter text with a top speed of over 80 words per minute. This corresponds to the performance rate of a professional typist using the QWERTY keyboard," said Antti Oulasvirta, Senior Researcher at Max Planck Institute of Informatics.

Moreover, the researchers trained a hobby pianist to generate text by learning the assignment of words and letters to notes. After six months of training, she was also able to enter sentences at up to 80 words per minute -- similar to the professional pianist but without reading from a music sheet. Now she can write e-mails and posts on Facebook faster than with the conventional keyboard, and at the same time she trains her piano playing skills.

With their study the researchers examined the question of why pianists can play notes on a piano twice as fast as professional secretaries can type letters on a keyboard. Therefore they investigated which factors of piano playing might improve text typing, in particular as it is done on input devices such as a so-called QWERTY keyboard.

Anna Feit and Antti Oulasvirta are researchers at the Max Planck Institute of Informatics. They conduct research in the field of human computer interaction in the German Cluster of Excellence "Multimodal Computing and Interaction." They investigate how music can play a useful role in the interaction between human and machine. Moreover, they want to know how humans can transfer their acquired abilities to new methods and which experiences they gain using novel computational applications.