Round Table - The Music Form Synesthaesia of Audiosurf

In a recent article on Offworld, Jim Rossignol wonders about the potential of games as "imaginative explorations" using the recently released HL2 mod about Karsakoff's Syndrome as a jumping off point. He writes:

Unfortunately I am going to have to be a bit poncy here because I want to suggest that Audiosurf -- a game made by an individual and not a university team of researchers -- is about exploring the sensations and experience of synesthaesia.

One of dozens of documented types, number form synesthaesia is "a result of 'cross-activation' between regions of the parietal lobe that are involved in numerical cognition and spatial cognition." In effect, the straight number line most people unconsciously imagine is twisted and sometimes even made three dimensional. It is a conscious form -- the number and its spatial position cannot be separated in the mind -- as well as constant.Audiosurf's tracks are "music form synesthaesia".

The algorithm that creates the 3D tracks out of an mp3 file is procedural but not random. The same track will be built every time, and players will come to associate parts of songs with hills and twists and valleys, just as a synesthete might always associate the number 13 with the same peak. We could then say that music form synesthaesia is "a result of 'cross-activation' between regions of the parietal lobe that are involved in aural cognition and spatial cognition."

There are over 60 types of synesthaesia, but, as far as I know, music form synesthaesia does not actually exist. And that's part of the point. Audiosurf is fiction that asks to explore and experience this muddling of the senses, to learn to associate music with space and become a synesthete for a few minutes. It's a kind of play I feel needs to be researched by academics and intentionally explored by game designers.

Addendum: In August 2008 a paper was published titled "The sound of change: visually-induced auditory synesthesia." It detailed the discovery of a new form where someone was "hearing sounds in response to visual motion and flicker." What would such a person experience when playing a game? Cacophony or music? Could a game be designed to emulate this form of synesthaesia and make it beautiful? Yes. It was called Everyday Shooter.