In this provided photo, University of Oregon senior instructor of audio production Jon Bellona works with students at a summer camp at the Eugene Science Center, July 2023. Bellona and a national team of researchers have transformed ocean data sets into sound as part of a three-year pilot project that explores how museums, aquariums and other informal learning institutions can make data more accessible.
Nic Walcott/University of Oregon Communications
When you go to a museum or visit a science center, it’s not uncommon for graphs, charts and other visual data displays to be included in the exhibit. But those displays can be inaccessible for blind and low-vision visitors. Researchers with the Accessible Oceans pilot project are exploring how to turn some of that data into sound so it’s accessible to more visitors. The project’s team of interdisciplinary researchers has been gathering feedback from visually impaired students and teachers, as well as ocean science experts, on how they can accurately represent ocean data in a series of sound presentations.
Jon Bellona is a sound artist and senior instructor of audio production at the University of Oregon, and Amy Bower is a senior scientist at the Woods Hole Oceanographic Institute. They join us to talk about the project and how they hope to make data more accessible.
The following transcript was created by a computer and edited by a volunteer:
Dave Miller: This is Think Out Loud on OPB. I’m Dave Miller. When you go to a museum or visit a science center, it’s not uncommon for there to be graphs, charts, or other visual data displays included in the exhibit. Those can be inaccessible for blind or low vision visitors. Researchers with the Accessible Oceans pilot project are working to change that. They’re exploring how to turn some of that data into sound so it can be appreciated by more visitors. Jon Bellona is a sound artist and senior instructor of audio production at the University of Oregon. Amy Bower is a senior scientist at the Woods Hole Oceanographic Institution. They both join me now. It’s great to have you both on the show.
Amy Bower: Thank you.
Jon Bellona: Thank you.
Miller: Amy Bower first, how did this project start?
Bower: I have been developing a vision problem for the last 30 years or so. And I was diagnosed after I started a graduate program to be an oceanographer. So all these years, while my vision has been declining, I’ve been searching for alternative means to access data. And data sonification is certainly one of those. So I’ve just kept my ear to the rail, so to speak, for opportunities to explore data sonification. And I got teamed up with Jon and a couple other interested folks, and we put in a proposal to the National Science Foundation to try out data sonification on some ocean data that would specifically be shown in museums.
Miller: Jon, from an audio perspective, what are the challenges of turning data into sound?
Bellona: Some of the challenges in turning data into sound is how it’s going to be perceived by a lot of different people. There’s so many different ways. That’s part of the fun, you get to design the data values and map them onto sound parameters. For example, if you’re mapping the values and say as the values go up, you make the sound louder, or the pitch will go up. And a lot of those decisions, it’s hard to know sometimes if the choices you’re making are going to be the best. And I think what’s great about this project is we’re doing this inclusive design process that’s paired up blind and low vision teachers and students, as well as scientists and educators together to try to create a best practice auditory display that includes data sonifications.
Miller: You’ve been as a team drawing on existing data to do this. You’re not doing original work right now with the purpose of turning that into sound. I want to play folks an audio clip that your team has created. This one is based on data that came from sensors measuring pressure on the ocean floor around an underwater volcano off the coast of Washington. And there are basically three main sounds that we’re gonna hear. There’s bubbling as the volcano’s magma chamber fills. There’s a whine on the seafloor that rises in response to that pressure. And then there’s a boom that I think is self-explanatory, that marks the volcano’s eruptions in 1998, 2011, and in 2015. The data set starts right before that first eruption.
[Sounds play]
Miller: What is the “boop beep” at the beginning, and the “beep boop” at the end?
Bellona: Those sounds we specifically designed as a result of this study, the indication that you’re about ready to listen to the sonification, it’s about ready to start. And at the very end, the sonification is over. It came through interviewing teachers at Perkins School For The Blind, and how they use sound in the classroom. They use devices that give a sound when they turn the device on, and when they turn the device off. And also in interviews with those teachers, we found that often when they’re listening to data sonifications in a classroom setting, and it ends, there is a long pause. And in that long pause, people are trying to understand, students and teachers, “is it over? When can we start talking again?” And so it was really clear to us in our development of this, we need to give a clear indication that the sonification would start, and that it was over, so that anybody, whether you’re sighted or non-sighted, you would be able to hopefully understand that you’re listening to quantitative information.
Miller: Amy Bower, did you get a chance to be around people as they were listening to this? I’m curious what the experience was like to be around, say, a young person or an older person who was hearing this data?
Bower: Yes, I have been. We’ve done some testing of these sonifications in various settings. And one in particular which Jon and I did together was quite interesting. Members of the general public came by, and we invited them to sit down and listen to the sonifications. And people afterwards would say how much they appreciated this mode of learning about oceanography or science. They’d say “I was never good at math, I hated science, I didn’t get it. But this, I get this.” It somehow broke through their perception that they weren’t interested in science, and got them excited. And that was really gratifying to see.
Miller: And Jon, my understanding is Amy is talking about people who might be blind or have low vision. Do you see potential benefits to this work, of having data turned into sound rather than turned into a visual representation for people who themselves are not blind or low vision?
Bellona: Absolutely. When we talk about using sound, the potential is to provide what we call “curb cut” effects. Just as you think about curb cuts at crosswalks that benefit more than just wheelchair users-
Miller: People with bikes, people with strollers, people who have any kind of ability issues.
Bellona: Sonification can make quantifiable data more accessible. For blind and low vision learners or researchers, sonification can provide access to quantitative information. But for sighted learners, sonification can provide another sensory opportunity to engage, to explore, and to communicate.
Miller: Let’s listen to one more clip. This is based on data from sensors off the coast of New England that measure how much carbon dioxide the ocean absorbs over time from the atmosphere, and also how much it releases over time back into the atmosphere. The CO2 absorption sounds like someone sipping through a straw, or something sort of slurpy. The CO2 release sounds like a rush of wind, and there are also voice markers for the seasons.
[Sounds play]
Voice: When we hear it all together, the ocean almost sounds like it is breathing. Breath in during the cold winter, breath out during the hot summer, absorbing carbon dioxide when it’s cold, and then releasing or outgassing when it is warm.
Miller: And that was obviously some added information. There’s some at the beginning of the sounds, and some after to help people understand what the data is showing them. I’m curious whether it’s a slurpy sound, my understanding is that it might be turned into something else to communicate the same numerical data. Do people sometimes mistake what they’re listening to? Do they assume that this is a recording of a natural process, that a microphone was put somewhere and you’re hearing what the world did, as opposed to hearing data turned into sound?
Bower: Yeah, once in a while that happens, someone will misunderstand what the sounds represent. But not most of the time. And it really behooves us to make sure we clearly explain that this is a representation of numbers, not a recording.
Miller: How do you hope that these tools could be used in museums or aquariums?
Bower: Well ideally, my dream is that this kind of data sonification becomes a routine component of any exhibit that has quantitative information, so that anyone can walk up to the exhibit, and if there’s a data graph or something as part of the exhibit, that they just push a button and they get to hear it instead of having to look and figure it out and scratch their head about a graph. A lot of people don’t really understand how to read a graph. And so to be able to augment the visual representation with the sonification I think would be very engaging too. Many people perceive information more easily and more interestingly through sound rather than vision.
Miller: Jon, the way I understand this, your whole team has been working hard to figure out ways to represent data and sound that that people will respond to and understand. This is real humans doing real work. Can you imagine a future where that work is automated in a way? Say, a graph on a website can be turned into sound automatically in a way that would actually be meaningful for somebody who’s blind who goes to that web page, the way that that text now can easily be turned into voice?
Bellona: Well, the cool thing is that some of those tools already exist. There are online tools that can turn data, graphs, numerical values, into sound. Some of the choices around them are kind of limited, and some of the tools themselves are not accessible. But I think we’re making some good strides into opening up or expanding data sonification for everyone. The possibilities are still fresh. The fact that data sonification as a field and an auditory display international community is still 30-35 years old, there’s a lot of possibility and growth, especially when we’re thinking about online digital tools that are currently happening right now.
Miller: Jon Bellona and Amy Bower, thanks very much.
Bellona: Thank you.
Bower: Yeah, thank you very much. Good talking to you.
Miller: Likewise. Jon Bellona is senior instructor of audio production at the University of Oregon. Amy Bower is a senior scientist at the Woods Hole Oceanographic Institute.
Contact “Think Out Loud®”
If you’d like to comment on any of the topics in this show or suggest a topic of your own, please get in touch with us on Facebook, send an email to thinkoutloud@opb.org, or you can leave a voicemail for us at 503-293-1983. The call-in phone number during the noon hour is 888-665-5865.