informa
3 min read
Blogs

Round Table - The Music Form Synesthaesia of Audiosurf

Unfortunately I am going to have to be a bit poncy here because I want to suggest that Audiosurf -- a game made by an individual and not a university team of researchers -- is about exploring the sensations and experience of synesthaesia.
In a recent article on Offworld, Jim Rossignol wonders about the potential of games as "imaginative explorations" using the recently released HL2 mod about Karsakoff's Syndrome as a jumping off point. He writes:

Whether or not it's actually educational, or simply fictional, what's interesting about this to me is that Korsakovia, and to some degree Dear Esther, is about exploring the nature of conscious experience. And that's not just a critic being poncy: that's what it's /explicitly/ about, as the stated intent of the project.
Unfortunately I am going to have to be a bit poncy here because I want to suggest that Audiosurf -- a game made by an individual and not a university team of researchers -- is about exploring the sensations and experience of synesthaesia.

300px-Galton_number_form.svg.pngOne of dozens of documented types, number form synesthaesia is "a result of 'cross-activation' between regions of the parietal lobe that are involved in numerical cognition and spatial cognition." In effect, the straight number line most people unconsciously imagine is twisted and sometimes even made three dimensional. It is a conscious form -- the number and its spatial position cannot be separated in the mind -- as well as constant.Audiosurf's tracks are "music form synesthaesia".

The algorithm that creates the 3D tracks out of an mp3 file is procedural but not random. The same track will be built every time, and players will come to associate parts of songs with hills and twists and valleys, just as a synesthete might always associate the number 13 with the same peak. We could then say that music form synesthaesia is "a result of 'cross-activation' between regions of the parietal lobe that are involved in aural cognition and spatial cognition."

There are over 60 types of synesthaesia, but, as far as I know, music form synesthaesia does not actually exist. And that's part of the point. Audiosurf is fiction that asks to explore and experience this muddling of the senses, to learn to associate music with space and become a synesthete for a few minutes. It's a kind of play I feel needs to be researched by academics and intentionally explored by game designers.

Addendum: In August 2008 a paper was published titled "The sound of change: visually-induced auditory synesthesia." It detailed the discovery of a new form where someone was "hearing sounds in response to visual motion and flicker." What would such a person experience when playing a game? Cacophony or music? Could a game be designed to emulate this form of synesthaesia and make it beautiful? Yes. It was called Everyday Shooter.

Latest Jobs

Treyarch

Playa Vista, California
6.20.22
Audio Engineer

Digital Extremes

London, Ontario, Canada
6.20.22
Communications Director

High Moon Studios

Carlsbad, California
6.20.22
Senior Producer

Build a Rocket Boy Games

Edinburgh, Scotland
6.20.22
Lead UI Programmer
More Jobs   

CONNECT WITH US

Register for a
Subscribe to
Follow us

Game Developer Account

Game Developer Newsletter

@gamedevdotcom

Register for a

Game Developer Account

Gain full access to resources (events, white paper, webinars, reports, etc)
Single sign-on to all Informa products

Register
Subscribe to

Game Developer Newsletter

Get daily Game Developer top stories every morning straight into your inbox

Subscribe
Follow us

@gamedevdotcom

Follow us @gamedevdotcom to stay up-to-date with the latest news & insider information about events & more