GO3: Masaya Matsuura On Music As Inspiration

The title of Masaya Matsuura's much anticipated talk at GO3 was quite obtuse: 'While waiting to cross, an airplane flies over the boxy heads.' It turned out to be entirely literal, as the first thing the GO3 audience was shown was a simple cell animation of such a scene, which was then repeated with two slight changes – a visual element (the plane was flying upside down), and an audio element (a slightly altered soundtrack). When shown separately, it was evident that the audio difference was actually the greater change, and this demonstration of the power of music led naturally to the sub-title of his address 'the inspiration of music on game development and its advancement into the future'. After a brief introduction and discussion of his musical and game production background, Matsuura launched into his talk proper. He recalled that in the 1990s when he was working on PaRappa, "hardy anyone was doing anything similar", but after the game's 1996 Japanese release "it became successful, many variations went on sale, and music based games became a widely recognised genre, at least in Japan." While the road has been tougher in the US, he complimented Harmonix (who he calls "my brothers"), stating that "in the last year we have [finally] seen successful music based games emerging in North America with the success of Guitar Hero", which Matsuura considers "wonderful." But Matsuura is still not happy with the state of game audio today, complaining that music in many games is simply background filler, and while the quality has improved it's role "is still like it is in movies." He mentioned that he had previously been quite negative about the cinematic aspirations of some modern games, but that now he feels that like everything, it can have a place. However, noting the sales trends in Japan, where the DS and 'lifestyle' games (such as his own Tamagotchi Corner Shop) have dominated for two years now, Matsuura stated, "In Japan, the industry is moving away from cinematic type games, and even the role of music has been changed." Back onto music-specific titles, Matsuura mentions that "Eventually Guitar Hero achieved success in North America, but it's still rare. So what is needed for this category to evolve further?" While he noted that some people think games need the involvement of "so called 'cool' musicians", Matsuura vented his disgust at commercial music, stating that MTV 'artists' seemed "more interested in sexy dancing then actually playing instruments." But he realised that without real musicians, games "could become a breeding ground for ignorance and laziness, where the music is belittled", a point he backed up by noting that in Japan, many video game soundtracks are not even registered for copyright. Matsuura complained that many soundtracks are created as an afterthought when a game is all but finished. In his opinion, "real musicians" can work on projects, but should also keep creating music for their own amusement, not just because they get paid. In a stinging barb aimed at professional musicians both within and outside the games industry, he brought up an analogy to the visual arts: "[In the visual arts] you must continue to create your own works. If you are creating only what has been asked of you, you are only doing production, not creating art." After a quick aside about game studios becoming independent to make the games they want to, adding "and I think this is a great thing," Matsuura continued. "Music is a mysterious thing – in our memory it clearly exists by itself, but in the outside world it never exists by itself, music has always existed with a strong link with things like people, places, parties, things like that - it exists only in a strong relationship with other things". He followed this train of thought by bringing up his personal studies in sociology, particularly the "primitive process of sharing," because, he believes, "we can find some clues to the future of music in the process." What followed were two videos of children engaging in singing and dancing games, the first was of a Japanese school group, and the second was of village children from Chad, in Africa. Both showed children singing and moving in non-competitive games, and while they were certainly culturally based, they were in many ways universal -- "you will see similar games all over the world," as Matsuura put it. He asked the audience, "Why would children do this? There are no clear rules, winning is not the objective. Why do people do things that at first glance seem meaningless, but enjoy doing so?" His answer came from an unlikely source – the 1991 book 'Peoplewatching' by zoologist Desmond Morris, an update to Morris' seminal 1977 work 'Manwatching,' and from it the concept Morris calls 'postural echo'. Morris observed the phenomenon that when two friends meet and talk casually, they automatically (but sub-consciously) hold a similar posture – Matsuura showed the audience examples on screen of some familiar looking scenes: people who were evidently friends, sitting and talking in a park. "Morris says the closer the friends are, the more similar their posture, eventually becoming carbon copies [of each other's physical shape]," said Matsuura, adding that this 'physical familiarity' is the key to understanding the children's games. "That is what makes the experience enjoyable – to learn a new rule and share that in real time," he said, explaining that because there is a level of risk involved (since you may fail), there has to be a certain level of positive emotion between the players before they try. It's easier for children, he believes, because "they don't think these things through before they act, they just follow the natural flow of those around them," but he believes that the inclination also remains in an adult's mind on an unconscious level, as demonstrated by the unconscious observance of 'postural echo' into adulthood. But what has this got to do with technology and games? Matsuura believes the same principle can apply, even when the other 'player' is not a human. He cited a Japanese film director who always adds blank frames to his scenes to imply blinking, and the audience always blinks on cue – 'postural echo' with a film. And he said it can be the same with games. "When we think of our interaction with AI, we usually think of text based communication, however, it is important to establish a non-linguistic connection, using postural echo." It was here that Matsuura thinks that most games have gotten their interaction with the player wrong. He said that in the 'adult real world', many things create negative emotion, such as situations of inequity. And in this situation, the worst case scenario is "the wall", where the person rejects the presence of the other. Matsuura said he was "very much annoyed at games based on such negative emotions," where the set-up is such that "you decide who is superior through physical attacks. It is an easy way to motivate gameplay, but if we continue to create such games, the future of this industry is doomed." Bringing it back to the topic of music in games, Matsuura wasn't hopeful about machines creating music by themselves, as the quality would be low, and AI can't understand emotional concepts. However, he looked at it from a different angle, asking "wouldn't it be good if machines could be used to create positive emotions?" He then showed us some experiments he had done with "high technology" in the pursuit of these machine-created positive emotions. He showed the audience a high-concept single-passenger car, designed to immerse the driver in technology in a positive way. Stating that we normally "arm ourselves" by getting into a large metal car, this concept vehicle gave the driver a more personal feeling of the car as his own body. Next, Matsuura brought up the experiments he has done with Sony's 'Aibo' consumer robotic pets. He passed the microphone over to his assistant, who introduced himself as Watanabe. Watanabe told us he was involved in the sound creation of the Aibo ERF7, and while the Aibo project has now been discontinued, some of the technology lives on. The 'pet' angle of Aibo allows people to approach technology with positive emotions, an effect he demonstrated with some photographs taken by the Aibo in the conference foyer on the previous day. This 'dog's eye view' gave a different picture of the world to regular photographs, as the portraits showed "positive emotions and softer expressions" then regular photographs of people's faces. He described the three month 'virtual life' of an Aibo as a 'three month song', with 300 individual segments in the song, any of which can come up at any time. Watanabe then demonstrated the musical abilities of the Aibo. First he played a tune on a keyboard, and had Aibo repeat it, which the robot did with accuracy. Next, he played a melody which Aibo was also instructed to repeat, but this time with it's own take on the rhythm, somewhat like the instruction a musician receives from the musical term 'rubato' ("could you hear the difference," he said). Finally, Watanabe instructed Aibo to perform it's own melody in response to what it heard from the keyboard, and to each of Watanabe's melodic phrases Aibo played back an original tune, which while still in the same key, and following the same basic pattern, was somehow very original, and even alien sounding in it's composition. Watanabe said that at this point in the process, "you start to think musically” about what you can play next, just to see what Aibo will do with it - "you become quite inspired." He said the technology is "simple compared to voice or image recognition, and yet it is a useful creative tool to humans." Watanabe passed the microphone back to Matsuura, who picked up the point. He said that recently developers have been "eager to imitate the real world" in games, but that "the important thing with games is not to virtualise reality with visual effects, but to virtualise people's fun experiences." He cited Nintendo's WarioWare and Brain Training games as good examples - and he said that taking the experience "out of the console and into the real world" will be one way to have significant meaning in the future. He cited the fact that even though Super Mario Bros was released over 20 years ago, its music is still played everywhere. "If you type 'Mario' into YouTube, you get about 100,000 hits" he said, and he gave us an example he found, an accomplished live performance of the Super Mario Bros theme played on piano, from the 2003 GDP conference. He said he had no idea where to place this in a formal musicial context, but that the desire was clearly there for people to express themselves in this way, and to mix the real world with their gaming experiences. Matsuura then gave a quick demonstration of this idea in practice – with his current project, a one button casual game called Rhythmica. A handheld game for mobile platforms, it plays with the user's own mp3 files stored on the hardware, and it creates a moving visual motif with them. While the song plays, "mysteriously letters appear one after another" on the screen, and the player must press the button the moment they see a letter that is part of the song's title. He then finished the presentation with a practical demonstration of all his theories, by playing a breathtaking duet with Aibo. Over a funky backing track, he played a keyboard and sang while Aibo amusingly danced. In the final section of the song Aibo performed the melodic 'call and response' of Watanabe's earlier demonstration, but in this larger musical context it was a spectacle to behold, and the audience furiously cheered. It was the perfect 'positive emotion' ending to an extremely in-depth presentation on the subject. Matsuura had made his point. [This report from the GO3 conference was published in association with PalGN, one of the largest Australian consumer video game websites, which is attending the Perth, Australia-based conference as a journalistic outlet and media sponsor.]