informa
7 MIN READ
Featured Blog

Rapid Iteration of Persistent Multiplayer Games

Screw the server and the database. Effective corner-cutting for indies making networked multiplayer games.

(Reposted from my game development blog)


This is a collection of thoughts on efficiently iterating persistent multiplayer games, based on my experience over the last couple years developing Antihero. The hilarious irony here is that my original development estimate for Antihero was 18 months; it’s now been two years and the game is still not done, so what do I know about making a multiplayer game quickly anyway. (At least I’m not dispensing advice on accurate scheduling of indie games.)

For the purposes of the post, I’m using “persistent multiplayer” to refer to networked multiplayer games that involve a client and a server, and that span multiple play sessions and therefore require persistent state management: asynchronous multiplayer games, Facebook-style “checkin” games, MMOs, etc. (Antihero is an async multiplayer game, which is arguably the simplest, architecture-wise, of these three.)

In game development, there’s always friction between having an idea and implementing that idea. Persistent multiplayer games compound the problem with more sources of friction in the form of an additional application (the server), and two additional data layers (the network and the database). Rapid iteration involves removing friction from the process; for Antihero’s early development, that meant removing the server and network from the equation as much as possible.

The primary forms of friction I’m concerned with here are serializationpersistence, and the authoritative server.(1)

Serialization

Game state in a non-networked game needs to be represented in one or two locations: in memory and - if the game supports saving - on disk. Networked games usually also involve a server and a database, potentially with their own data formats.

When you’re iterating a game, your data is constantly in flux: in a single-player game, if you want to add a mana property to your Hero, you just add a field on your Hero class, and you’re done. In a multiplayer game, you need to modify your client’s Hero class, modify your server’s Hero class, perhaps modify a Hero table in your database, and make sure that changes to Hero.mana are propagated everywhere.

Choosing a wire format - the format that your client and server use to send data to each other - is one of the first architectural decisions you make when developing a multiplayer game, and the choice has big ramifications on your ability to quickly iterate the game’s data structure. When I’m prototyping a multiplayer game, what I really care about is:

  • I can send data between the client and the server.
  • Serialization is maintenance-free (i.e., when the “mana” property is added to the Hero class, Hero.mana should automatically be included in serialized Hero objects).
  • My data classes are statically typed.(2)

Nothing else - serialization speed, data size, security, etc - matters: iteration efficiency trumps all other concerns. (I’ve worked on multiplayer games where there was an insistence on doing things “properly”, security-wise, up front. This bogs down the pace of iteration now for an insignificant potential payoff later.)

For Antihero, I chose Google’s Protobuf as my wire format. It satisfies my three prerequisites, and has Python and ActionScript implementations. (And, happily, it’s also fast and efficient.) Protobuf relies on code generation to work - you write your data structures in a simple description language and then a parser spits out class files in the language(s) you’re using. (Some devs are allergic to code generation, because they believe it increases maintenance costs and code complexity. They are wrong.)

If you’re not down with Protobuf, there are lots of reasonable-looking alternatives out there (ThriftAvro, etc). All the cool kids write their own serialization format! But you, dear indie, almost certainly should not; instead, just take somebody else’s and get on with your game.

Persistence

Alongside wire format, you’ll also probably choose a persistence layer early on. “Persistence layer” usually means “relational database of some sort.”

But a database is another source of friction. It imposes unnatural additional syntax on your server (in the sense that manipulating data in SQL is very different from manipulating a runtime object); it slows down refactoring, because database schema changes are a pain and similarly unnatural; and it’s another frigging service to set up and futz with.

I avoid the database for as long as I can, and Protobuf actually helps here as well, because it’s trivial to save protobuf objects to disk. In the early days of Antihero’s development, I created protobuf_table.py, a simple Python class that the server used to manage a collection of protobuf objects (one for each match in progress), save them to disk when they changed, and load them all back up on startup. With protobuf_table, Antihero had many happy months of development where the client, server, and pseudo-“database” were all using the same data format. (And even now that the game has a proper database for account management and whatnot, its “active_games” table primarily consists of just a “data” column that stores the games’ protobuf data directly as a binary blob.)

Authority

I don’t worry about doing the right thing, with respect to security, during prototyping (and, possibly, beyond). The “right thing” is to assume that the client is in the hands of an enemy that’s tampering with it, and therefore to have the server validate everything. But this is a massively premature optimization (especially when the client and server are written in different languages, necessitating that game logic be written twice).

One game I worked on, in a past job, was still far from being fun - let alone shipped - when it was decided that server validation was necessary to make the game more hack-resistant. This took several programmer-weeks, slowing the game’s development without, of course, making it any more fun. And when the game launched, it still wasn’t fun; it never found traction and was written off as a failure and shut down shortly afterwards. Its low quality was what truly made it hack-resistant: it never had enough players to be a tempting target to exploit in the first place. All that time spent making the game less exploitable was simply wasted.

The Antihero server, whenever possible, acts as a dumb message-passing mechanism. Data comes in from a client, gets stored, and then naively shipped back out to other interested clients; aside from making sure that an out-of-date client isn’t sending broken data, no validation is performed. And when the game ships, the server probably won’t do a whole lot more; since the game will probably fail anyway, I won’t spend time on server validation unless and until I get lucky and the game finds a player base.

A Client-Centric Philosophy

My approach to multiplayer development can be boiled down to “spend as much time as possible working on the client.” The client is the game, as far as the player is concerned - everything else is an implementation detail. With Antihero, I’ll often go for weeks without touching the server except to recompile its protobuf files.

A solid server is only a necessity for a finished popular game; prototypes are by definition not finished, and the brutally hit-driven nature of the marketplace ensures that most games will never be popular. Having to shore up a janky server for a launched game that’s found traction is the very definition of “a nice problem to have.” Server procrastination is a virtue - much better to spend your precious development time increasing a game’s chances at popularity by making it interesting.


  1. A quick aside on programming languages. The Antihero client is written in ActionScript, and the server in Python. I arguably ignored a huge potential source of efficiency by using different languages, but I wasn’t crazy about the alternatives. Three Rings, my former employer, uses Java for both the front-ends and back-ends of Puzzle Pirates and Spiral Knights, its big MMOs; client and server share lots of code, and no game logic needs to be written twice. But writing a Java game for iOS - while possible - is kind of a pain. Unity, with a Mono-based server, was another option, but I went with ActionScript/Python since I already had a bunch of reusable code in both languages, and also because “Flash in the front, Python in back” is so close to being a middle school-quality double entendre.

  2. During prototyping my data structures are in constant flux; static typing - as opposed to, say, using untyped collections of key-value pairs, like you’d get with JSON - means that refactoring is fast and easy rather than obnoxious and hateful.


(If you’re interested in running your own thieves’ guild, you can read more about Antihero, my in-development game, or follow me on Twitter.)

Latest Jobs

Manticore Games

San Mateo, California
8.23.22
Senior Software Engineer - Mobile

Sony PlayStation

San Diego, California
6.23.22
Sr. Online Programmer

The Walt Disney Company

Glendale, California
8.1.22
Associate Marketing Manager - Walt Disney Games

Insomniac Games

Burbank, California
8.26.22
Accessibility Design Researcher
More Jobs   

CONNECT WITH US

Explore the
Subscribe to
Follow us

Game Developer Job Board

Game Developer Newsletter

@gamedevdotcom

Explore the

Game Developer Job Board

Browse open positions across the game industry or recruit new talent for your studio

Browse
Subscribe to

Game Developer Newsletter

Get daily Game Developer top stories every morning straight into your inbox

Subscribe
Follow us

@gamedevdotcom

Follow us @gamedevdotcom to stay up-to-date with the latest news & insider information about events & more