EWU Professor researches How to turn numbers into music


Abbi Vance for The Easterner

By John Corley, Contributing Writer

People may listen to a piece of music and interpret it as a reflection of the musician who created it. But what if music could be more than just a glimpse into a person’s soul? Could music also be a reflection of the weather, traffic patterns or any other scientific measurement of information? A professor at EWU, along with a few other researchers, are trying to answer that question right now.

Jonathan Middleton, a music professor at EWU, assisted in a study researching how music, or sound, can be used to better interpret scientific data. Middleton, along with a researcher from the Francis Crick Institute in the U.K., published research on music and data in the  online journal, Heliyon.

In his office in EWU’s music building, Middleton demonstrates a piece of music based on weather patterns by using a music algorithm program. Numbers are entered into the program and are used to determine what musical note is played. The program can also determine whether the notes are played in a staccato or legato style.

 The music Middleton created is based on 10 consecutive days of weather pattern data in Finland, involving temperature and cloud cover. Middleton constructed his piece in a relatively simple manner; he uses a low flute sound to represent weather temperature and a synth keyboard sound to represent cloud cover or sunlight.

Initially, as the music is played, only a few notes from the keyboard are heard within a second or two of each other, meaning low temperatures and plenty of cloud cover early on in the 10-day weather pattern. Each note is played in a legato style, as each note carries into the next.

As the music progresses, the keyboard sounds, or temperature, starts to speed up, meaning the weather is warming up. Flute sounds, or sunlight, in the background of the musical piece also speed up. The flute is also played in a legato style. The music goes through fluctuations of speeding up and slowing down, representing standard weather patterns of temperature changes.

The weather data musical piece that Middleton created is an example of his on-going research of turning data into music. Sonification is the term Middleton uses to describe “interpreting data through sounds rather than through visual display.”

Middleton has researched sonification since 2004, and he developed software that can turn numbers into musical melodies. Middleton said he wanted a different way to show students how to compose music, but he also wanted to ask questions like “what does pi sound like?”

Courtesy of Eastern Washington University

“A good example of that would be the Geiger counter, where you hear a series of intense clicks as you get closer radioactive material. The intensity of those clicks is quite informative and maybe more informative than a needle moving,” Middleton said.

Middleton’s software developed in a collaborative effort between him and students in the Computer Science Department.

Chris Peters is a senior lecturer in computer science at EWU, and he said one of the last required courses for computer science students before graduation is to work with an EWU professor on a project.

“There have been modest advances and frustrations in that regard, all at the hands of senior project students and Dr. Middleton’s input and patience,” said Peters, referencing the collaborative effort between Middleton and computer science students developing data-analyzing music software. “[Dr. Middleton’s] a true educator.”

For the research published in Heliyon that Middleton describes as the protein sonification project, he said the goal was to help scientists hear patterns or outliers in their data. Middleton said the process to turn data, such as protein information, into music is very collaborative because “he doesn’t know proteins so well.”

“I kind of have to rely on whoever I’m working with to tell me what’s meaningful and what’s not meaningful,” said Middleton. “They’ll send me data and I will make sounds from the numbers, or letters, depending on the form of the data, and I’ll ask them ‘is this meaningful to you?’”

Middleton said a method scientists use to allow him to interpret their data is presenting it in tables with columns he can understand but that this collaborative process is still in its very beginning stages.

“I’ve been working on this project and all I can say is the ship has sailed; we haven’t arrived anywhere yet,” said Middleton. “The potential is quite intriguing; we could get to a point where abnormalities in protein structures could lead to discoveries of diseases, but we’re not there yet.”

Robin O’Quinn, an associate  biology professor at EWU, said she has a hard time wrapping her head around situations in which sonification could be more effective than visual data. She did say, however, that she could have an open mind about it.

“I think scientists work pretty hard to convert data to the visual modality that best communicates the most information,” said O’Quinn. “So from my interpretation of what [Middleton’s] doing, he’s offering an aural way of expressing data. It’s obviously easier for me to wrap my head around a bar graph that displays a comparison between two means, where there’s a direct application of the data. [It is] much harder for me to understand how he’s converting data to music, but I’m open to knowing more about his process.”

Middleton points out that sounds are used in the real world everyday to boost awareness of information, such as when metal detectors sound an alarm when they detect metal, or when crosswalks beep to inform the  visually impaired they can cross the street.

“We have sounds all the time we’re not always conscience [of],” said Middleton. “We’re taking that idea into the world of data and science for analysis.”

Middleton’s protein research is the first of at least three or four phases he is working on with other researchers to better interpret data through sound. For now, people will just have to be satisfied knowing what getting caught with metal in their pockets through airport security sounds like.