Like a lot of middleware developers, Tony Cannon started developing his own tool to solve a problem he himself was having. Unlike a lot of middleware makers, though, Cannon’s creation grew out of his problems as a player, not as a developer. As a pro-level fighting game player and one of the organizers of the Evolution tournament series, Cannon was worried that the arcade culture he was steeped in was deteriorating. "In the mid-90s, arcades were really dying, and we were in danger of losing this thing we really cared about, because everyone played in the arcades," he said in an interview with Gamasutra. A ray of hope came in 2005, Cannon said, when Capcom announced it would be releasing a console version of Street Fighter II: Hyper Fighting with online play that could hopefully reconnect players who were no longer able to meet at arcades. When the game came out, though, Cannon said lag and glitches made the internet play unusable. "It was just bad," he said. "It was literally unplayable for a hardcore fighting game person. If all you remembered was Street Fighter II on the SNES and you just jumped back in after eight years and just started playing it, maybe it was good for you, but for hardcore fighting fans it was just unplayable." Cannon started thinking about how he might fix these problems for timing-dependent games like his favored fighters, and came up with an idea he calls rollbacks. Instead of inserting any internet latency between the button press and the start of the move, his technique puts the lag on the beginning of the opponent’s move, with filler animation to hide the effect somewhat. "That’s actually way better. You might see a little glitchiness, but all your timing is the same," he said. After eight months of work, the first version of GGPO (short for "good game, peace out," a common post-match greeting in the fighting game culture) was ready by late 2006. The next step, Cannon says, was getting someone to notice, which was easier said than done. He approached Backbone Entertainment -- which was working on a console port of Puzzle Fighter -- with a description of his procedure, but the process went nowhere. "I learned something very important... a description of how it works is not enough to convince someone," he said. A better strategy, Cannon said, was simply layering his code on top of an arcade emulator called Final Burn Alpha, with support for some of his favorite older fighting games. That way, when Capcom’s Seth Killian was able to set up a meeting with the Street Fighter II: Turbo HD Remix development team in late 2007, Cannon was able to show them a demo of Street Fighter Alpha 2 and let them play live against someone from a player from Europe. The demo helped everything click in a way a simple description couldn’t, Cannon said. Backbone ended up using its own homegrown, rollback-enhanced net code for HD Remix, but the release still validated the concept used in GGPO, Cannon said. While other companies could do the same thing, Cannon says getting a license for GGPO also brings along months of iterations and optimizations he’s made for a variety of national and international internet connections, as well as testing feedback from thousands of Final Burn Alpha players. "That’s the real value of GGPO, I think, the concept as well as the quality of the implementation, which is very hard for developers to get in-house," he says. “How do you test your game out there on the internet for a year before you release it? You can’t.” Use of GGPO in commercial releases is accelerating, Cannon says -- the code has been licensed for Capcom’s retro revamp Final Fight: Double Impact, Namco Bandai’s Japanese arcade fighter Dragon Ball Zenkai Battle Royale and upcoming indie 2D fighter Skullgirls, as well as several unannounced titles. But whether they use GGPO or are just inspired by his lag-masking technique, Cannon says he’s happy developers are finally getting smarter about their net code for fast-paced competitive games. "That’s been my goal from the start, to get fighting games incorporating some technology that replicates the arcade feel so I can play these games online, because I don’t have time to go to arcades anymore," he said.