Labs

Data Sonification

D

ata visualization has driven the innovation of how we use and digest data over the past decades. A lesser-known technique of this is data sonification, the transformation of data into sound. Listeners are given a different perspective to identify patterns and relationships in data sets that can be better heard than seen. The most common uses for data sonification are building data exploration tools for the visually impaired and creating informal learning environments in museums and zoos. Still, the subject's uses and projects continue to grow.

Experimentation with Max MSP

I explored data sonification using Max MSP, a visual programming language used for sound, graphics, and interactivity. In the first experiment, I turned visual data from my laptop camera into sounds. The program would analyze the most dominant color, convert the color data to a frequency, and then convert it to audio. As the user moved about the camera, the pitch would change, resulting in changes in tone and complexity of the piece. In this interactive audiovisual space, the user uses the tools to create a synesthetic stimulation to hear colors.

I became more interested in learning more about sonification and other artists that have also explored this. It was then when I stumbled upon an article that described how Aphex Twin hid his face in one of his tracks. The image was only viewable with a spectrograph program, a tool that can view and analyze the contents of a music or audio file. In his example, the hidden face was easily masked as ambient music on the track and wouldn't suspect a listener.

The Aphex Face: Visualizing The Sound Spectrum From '#2' - Magnetic Magazine

I was determined to 'sonify' my face into sound using Max MSP with this concept in mind. After I uploaded an image to the program, I was able to break the image down pixel by pixel and follow a similar concept from the previous project where I converted that pixel data into a sound. After the first pixel was sonified, the program would move onto the next pixel and then repeat. While the overall project was successful, the noise that results is not an easy listening experience. Many listeners were confused since the sounds are abstract or ambient and require a trained ear to decode.

These ambient sounds have real-life applications and work well in the monitoring of processes. In a project by LARAS Sonification Laboratory, listeners can monitor network activity and detect unusual behavior. The differences in noise alert the listener of hacking or abuse disturbing the network. Another tool called Glide Radio allows you to monitor your own back-end systems this way.

Even though the popularity of data visualization is still on the rise, the tool is limited. While data visualization has to be manipulated to fit a screen, the sound always sticks to the data. Even though data sonification is still an emerging tool for pattern recognization and explorations in data, its possibilities are endless.

Amanda Esposito
Lead Full Stack Engineer
Labs

Ticketing Kiosks: Should Your Business Build It or Buy It?

Amanda Esposito

Lead Full Stack Engineer

Do you need to buy a ticketing kiosk or should you build one in-house? Learn how to choose the right ticketing solution and what goes into the build vs buy analysis.

Read More
Fireside

Clutch Lists Wyrmix as a Top Internet of Things Company in Texas for 2021

Mark Maxey

Interactive Designer

Wyrmix was awarded the "Best B2B Firms in Texas" award from Clutch.co

Read More
State of the Industry

When Customers Prefer NFC Over QR Codes

Alexander Melton

Founder and CEO of Wyrmix

Customers, in general, prefer NFC-based technologies when social influence and speed are primary factors in the customer's decision-making process.

Read More
State of the Industry

Residents of Poor Neighborhoods Pay 12.2% More For groceries

Alexander Melton

Founder and CEO of Wyrmix

A new study released in May published in the American Economic Journal has shown a causal link between increased grocery prices in poor neighborhoods and distance from the city center.

Read More
Labs

AR in Fashion

Amanda Esposito

Lead Full Stack Engineer

Mixed reality experiences are slowly taking over the fashion world, and so too is Pokemon. Pokemon Go, Gucci, and North Face collaborated to create an AR location-based experience for their customers.

Read More
Labs

Data Sonification

Amanda Esposito

Lead Full Stack Engineer

Data visualization has driven the innovation of how we use and digest data over the past decades. A lesser-known technique of this is data sonification, the transformation of data into sound.

Read More
Insights

Where does Experiential Marketing fall into the world of Phygital?

Akash Arora

VP of Strategic Partnerships

Scream loud enough via a flyer, a billboard, or even a commercial, and you’re bound to get someone’s attention. Unfortunately, those methods have been exhausted.

Read More
Insights

Smart Retail — The Future of Retail is Shaped by Disaster

Alexander Melton

Founder and CEO of Wyrmix

Many retailers have disappeared, been called in or forced to file for bankruptcy. The future of smart retail is here whether we like it or not.

Read More