These Open Source and Cloud Technologies Power Nintendo’s StreetPass Relay

Bryan Cashman is the founder of Callvention, a service enabling phonecalls with game developers.

Nintendo is expediting the release of online services through the adoption of open source and cloud technologies, at a speed the company could never have performed with only in-house tech.

In a discussion below with Nintendo President Satoru Iwata and members from Nintendo's Network Development & Operations Department, the company shared its success using free open source tools Hbase, Puppet, and Fluentd.  The company also highlighted the advantages of coding in Ruby, as well as the ability to move fast using AWS, Amazon.com’s cloud architecture.

Building a Global Online Product

The discussion took place to celebrate the release of StreetPass Relay, a cloud service that transmits data to Nintendo 3DS handhelds whenever customers walk past one of 100,000 global internet access points (in the US, this includes AT&T wireless locations, as well as McDonalds and Starbucks restaurants). When gamers walk past these access points, StreetPass gameplay data is uploaded to Nintendo’s servers, and a previously nearby player’s gameplay data is downloaded directly to the user’s 3DS.  Data transmitted includes gameplay data from the 3DS’s library, including ghost races from Mario Kart, virtual homes from Animal Crossing, and puzzle pieces from Mii Plaza.  

Hbase

To manage the constant flow of data for StreetPass Exchange, Nintendo turned to Hbase, a data management tool also used by Facebook and other social networks.

“HBase can handle high amounts of processing, while being good at increasing system performance by adding on servers,” noted Taisuke Kawahara, of Nintendo's System Development Division.  “I thought it was suited for our scenario where data would be processed constantly, and how we could easily build upon it to expand the infrastructure when we needed to depending on the workload.”

Hbase also supported a flexible processing model, where the company could expand infrastructure when needed.

Puppet

Nintendo has three years of experience using Puppet, and reuses prior settings with each new project.  By adopting prior settings, Nintendo was able to launch StreetPass Exchange within three months of starting the project.

This already established work allows Nintendo to launch products quicker.

“You prepared ahead of time so we could begin this project any time,” said Nintendo CEO Satoru Iwataa to his team. “To start a large-scale network service you first have to configure multiple servers to work together. Managing that configuration takes a lot of work.”

“We were able to release this three months after starting the project because we adopted Puppet ahead of time,” said Kawahara.

fluentd

But with millions of 3DS users using the service, and 100,000 different access points, Nintendo needed responsive analytics to quickly identify any user experience challenges.  The company turned to Fluentd, an open source big data solution that aggregates access logs in real time.

“That made it possible to grasp the entire situation with a time lag of only about one minute,” said Masatoshi Yamazaki of Nintendo's System Development Division.

“By using Fluentd, you can understand in a flash what kinds of users are having StreetPass encounters, how many hits there are, and at which access points,” noted Iwata.

Powered by Amazon

But none of this would be possible without Amazon.  Nintendo utilized AWS, a dynamic cloud service offered by Amazon, to host StreetPass Relay.

“It’s impossible to imagine starting this kind of service in such a short time without a cloud,” said Yamakazi.

“Using the cloud as servers is very suited to making a service when, as with StreetPass Relay Points, you cannot predict ahead of time how many hits will come in from the 100 thousand access points around the world,” said Iwata. “This way, you can easily increase the number of servers in line with circumstances, or decrease the number if they aren’t necessary.”

Iwata also confirmed that Nintendo heavily leveraged cloud technology in the past for Miiverse, the social network built into Wii U, the company’s latest home console.

In the end, the responsiveness and dynamic flexibility of the cloud made AWS critical.  “If there’s a lag, the players will leave,” said Iwata.

Over One Hundred Members

The project was built by Nintendo’s Network Development & Operations Department, home to over one hundred employees, most of which are engineers.

Nintendo’s engineers code in Java, Perl and Ruby, but for StreetPass Relay, the team relied on Ruby.  “We needed to begn service as quickly as possible,” said Kawahara.  Ruby is known as an agile programming language that enables developers to quickly build web applications.

Source: Iwata Asks

The author, Bryan Cashman, blogs about business issues in the video game industry at CONSULGAMER.