Sonification of Math Graphs for People with Visual Impairments
About this website
This website is a project that uses sonification (i.e. sound) to communicate mathematical graphs to people with visual impairments. Math graphs are commonly used in education. But, since they are based on visuals, it is not easy for people who are blind to know the content of the graphs. In this project, I want to explore how sonification can be useful to communicate mathematical graphs in real time using only a web browser.
More secifically, I focused on the mathematical concepts of continuity and discontinuity taught in calculus. In general, math graphs are frequently used to introduce the concepts of continuity and discontinuity. But, what if you can't rely on visuals? In this project, I designed sonificaiton to introduce the concepts of continuity and discontinuity.
Content
The content consists of three parts: 1) Introduction to sonification, 2) Continuity and discontinuity, and 3) Quiz.
1: Introduction to sonification
I use two types of sounds: pitch for y-value and click sound for x-value.
The first is to express the change in y-value with pitch; it plays the lower the pitch when the y-value is smaller and plays the higher pitch when the y-value is larger. Note that the pitch alone does not tell you the specific y-value (i.e. if y is five or ten), but only the change of the y-value (i.e. increase, decrease, stable, etc).
The second is to express the change in x-value with click sound like a metronome. It plays a click sound every time the x-value moves by one in about one second. Because the range of x-value in my graphs is fixed from minus five to plus five, you will hear 11 clicks in total starting from x equals minus five to x equals plus five in about 10 seconds.
To give you an idea of what sonification is like, I present 6 examples of math graphs.
2: Continuity and discontinuity
Avoiding rigorous mathematical definitions, we can simply say that a graph is continuous when you can draw it in a single stroke without holes or skips, otherwise a graph is discontinuous. To describe that in sound, a continuous graph sounds uninterrupted from the beginning to end (i.e. no silence) and with no jump in pitch.
With sonification, I present three common types of discontinuity: removable discontinuity, jump discontinuity, and infinite discontinuity.
Removable Discontinuity
Jump Discontinuity
Infinite Discontinuity
3: Quiz
I present 5 quizzes to ask the concept of discontinuity. I hope you enjoy!
About me
As of April 2021, Keita Ohshiro is a master's student in Integrated Digital Media program at New York University. This project is a part of my master's thesis. For more information, see Keita Ohshito's website.