Analysis: 'Socially Awkward' - It's Beginning To Look A Lot Like Metrics

[Continuing his series on social games, game designer Patrick Dugan explains why metrics are both are "the Promethean firebrand that sears open the chains of blind intuition", and "a trap!" Here's "Socially Awkward V".] Why is Zynga valued at almost $20 beeeelion dollars? Because theirs is bigger. Their audience, that is. They'll be the first to tell you: they measure anything, and fundamentally, that is why they are so big in the first place. It's a crass inversion of the Heisenberg Uncertainty principle -- the more they measure their potential wealth, the more quantitatively real that wealth seems to become. One billion dollar run rate. It's going to be a hell of an IPO. I can't wait. Used to be that designers would have very large egos; they collectively accumulated this sense because game design involved lots of intuition. Intuition really means pulling something out of your ass -- but you happened to have eaten genius for dinner last night. It's not accountable, and for a long time, neither has game design been accountable as a profession. Before it could get that far, it had to make a pit-stop in college, where it piled on PhD dissertations about video games like freshman pile on Busch Lite. Then game design got an MBA, and it started slinging engagement to users of all stripes. Many young people who are enamored with games and want to grow up to be game designers look to level design as their first step. I used to draw on maps when I was in 3rd and 4th grade. I randomized my maps, except, it wasn't truly random because the human brain is incapable of truly random thought (so are computers). Instead, I would draw the outer limit box, then trace L shapes on opposing corners, starting at a point chosen by whim but not entirely. The converging snakings of these corners were wound together until the maze was done. I "procedurally generated", by hand, about a hundred maps and connected them into a bigger RPG, but it was hard to run it with just one other player. JRPGs almost ruined me. In 6th grade, I graduated to Warcraft 2 editing, then StarCraft by 9th. StarCraft came out when I was in 7th grade; the two-grade jump is because: a) Myth, Might and Magic VI, Half-Life, Fallout 1 and 2, Baldur's Gate and System Shock 2 were released in that interval. b) I was intimidated by programming until I played System Shock 2 and tried the hacker mini-game, and figured if it was like that, I would be ok. StarCraft's "triggers" system wasn't really that much more complex, now that I think about it. After level design, they step up to trying to codify a deeper design that encompasses the core rules and systems of a new kind of game, rather than just permuting content in a genre you liked (but no longer care for). This is the awkward stage; you have this choice between getting laid or running a D&D campaign, then writing your own book for a new tabletop role-playing game based on futuristic conspiracies. That game was bad ass. Before that I designed and ran a Matrix tabletop game. Say whaaaaat? Then you try to conceive of really experimental stuff. I was in this phase from 19-23. Graph paper and cheap coding engines, board games and whiskey, Deleuze and Guattarri. It got beaten out of me when I started doing social games. Heh, well, not entirely, eh? It got beaten in any case. This is when you learn about metrics. After all of these steps, I got quantitative, never completely abdicating my intuitive strengths but backing them up with math. This is when I stopped trying to be an auteur and sold my time for a professional rate instead. Having kids makes you do that. In order to put the place of metrics in the history of game development into its proper context, you need console game experience for contrast. You need to have a faded but not forgotten memory of at least 50 time-wasting meetings where people couldn't agree on design choices that weren't really in their purview to make anyway, or where a designer is at odds with the team about the best priority or direction. Because these games cost millions of dollars each and take at least a year to make, the whole thing has high stakes -- your life will be greatly affected by any design decision, for better or worse. With metrics, you make the feature in a few weeks at most and then get feedback based on the statistical patterns of thousands of people within an hour. What a rush! Why waste time ever again? Metrics are the Promethean firebrand that sears open the chains of blind intuition. They are also the chains that blind the firebrand of Hephaestusian creativity. Yes, look it up, it's Greek mythology. But one thing is certain: when I graduated to doing game design by spreadsheet, I began to see significant economic leaps made in my personal situation. Goofy, unfocused creativity only made me $8500 a year, but being able to weave numbers brings you up to the 100k p.a. level. As if you can solve the problem with even more numbers. Once when I was in my experimental phase, I somehow got the idea, after tripping all night and contemplating concepts from Godel, Escher, Bach: An Eternal Golden Braid, that I might be Kurt Godel's reincarnation. We were both hypochondriacs; it makes sense to me more and more as the years pass. Now on Halloween, I dress up in a white suit and try to find a friend who will dress up as Einstein. Then I make a blood sacrifice. I was that guy on the left, before I died. Well I'm alive again, and I'm here to tell you, undecidable propositions are still inevitably produced in any formal system with a finite number of axioms. So get over all your earnest glow about this metrics empire we've built up around us, or you will alienate your closest allies. As Admiral Ackbar would say: "It's a trap!" The numbers are making us number. Number and number and number. Around and around it goes, clicking 1.4 times a second for 87 clicks per session of an average 2.4 sessions a day over 450 thousand daily users... USERS!!!!!!!!!!!!!!! Get ready to take your next hit, soccer moms! The confirmation bias strikes again. Every one is trying to see what they already wanted to see, and then, pop, whaddya know? They see it! I'm both split and tested on the matter. It's all about finding that point of pricing where you get the greatest net revenues; if you price higher you'll get fewer people but more money per person -- at what price do you reach a diminishing return? That's become a science. Early game flow has also become something of a science, with each consecutive click measured, showing where you lose people. If you're losing people, change it, maybe you get more people committed. There are statistical principles at work that demand certain design choices. Godel's Theorem proves that whatever matrix of numbers you throw up, there will arrive a crease in logic, a rupture of your frame of reference, where an undecidable proposition will be produced, telling you neither true nor false. The way that kind of thing tends to be digested in the corporate world is simple: it isn't. Then the apparatus reaches its point of perfect adaptation, to live for a quarter or two in glory, and then declines before being put to sleep. Zynga and those who model themselves on its example are like the Nuclear Power industry. They're really popular these days right? Ah, it's a cheap shot, but I'm not being paid by the word so let's take it. They process data and leverage chain reactions in clusters of high energy people particles. The data is the uranium, the game loop is the reactor, the BB-summary-write-query-visualization-crossRef infrastructure is the reactor antechamber where energy is converted into a usable resource. Most of the time, it's glorious, like 2 Giga-Watts of capacity is glorious, each million DAU a golden goose lined up in cages. But sooner or later your users' behavioral probability distribution will go kurtotic, the reactor will melt-down, and your goose will be, well, cooked. Here's another metaphor for you, since we're getting to the part of the essay where I have to follow up with something constructive. Let's say you were a scientist, a behavioral psychologist maybe, serious enough not to think B. F. Skinner is an odd name. You set up an array of sensors in a rat maze. So far, so good. You run millions of rats through, you know exactly how many pellets they take, how often, with how much navigational difficulty. But you want more, you want to be able to statistically model their behavior and know with a high degree of certainty what path a rat will take. Someone I know actually had this problem with a social game; they wanted to amplify the resolution of their model to a predictive extent. I said: "You're trying to solve the wrong problem, you're looking too closely and not at the big picture. Instead of trying to finely measure a maze, give the people a garden and figure out how to model it with a decent enough resolution that you have the same Business Intelligence capability that you have now with these stupid fucking rat maze games!" When your data gets non-linear, you need to innovate with the back-end architecture to be able to make meaningful turn-around times on analysis. Zynga uses Vertica to crunch graph analysis and identify the highest impact USERS as they spam other USERS into USING. I know this because it was in a white paper. That's cool, that probably helps with retargeting, getting the cloud of people to pulse like a vibrant plasma as active USERS USE each other. Every successful social game has operated with a brittle nozzle of USER-generated content and charged the exploration of that content sea with incentives to drive retargeting - high resolution graph analysis can help you do that. But what if you turn the nozzle back just a little bit more? Will your apparatus hold? All the really good BI people I met don't really care that much about BI, they're more interested in crazy shit like abstract algebra or Hopf maps or Bayesian networks. It's going to be a really fun field, because we're going to have to figure out how to answer questions about the rate, tenor, contour, multiplayer orientation and difficulty level of player-created content beyond weakly interactive decorative sprites arrayed on a 10x12 grid. Skill is one domain of player-created content, if you really think about it. A highly variable output with mathematically defined limits; sounds like a feast to me! Now I'm bringing the most new-school game design stuff full circle back to the old stuff, but anyone who has played a lot of Spelunky already knows what I'm talking about. Modeling skill-based performance in the context of social game economics is a challenge I'm approaching on a project. Basically I'm going to try and define a probability outcome and take an average, knowing that if the process must be repeated hundreds of times the average will be economically meaningful. Not sure how else to do it, would love for someone to tell me. It's easy to have Znyga envy. After all, none other than Justin Timberlake has informed us that a million dollars is not cool, and I guess a billion isn't even that impressive; now you need to shoot for $20bb and a dog. But I say: fuck Justin Timberlake. Seriously, fuck that guy. It's time to bring gameplay back. If you model your game from the bottom up, a priori, you can produce values that are reasonably accurate global maxima, rather than extremely accurate local maxima, at like, a thousandth of the expense of licensing Vertica and building a state-of-the-art BI department. It's also possible to model things that can't be deterministically derived by rendering them as constants, things like how frequently people would do something, how well they perform in a contest of skill -- these things allow you to ask very specific questions that are much cheaper to answer than general questions from the top down. Data can be sliced a billion different ways, correlations can be found in anything if you split the hay straws finely enough, but if you need very specific numbers it's more like finding a needle in a haystack with a metal detector. Or, if you prefer a more social analogy, it's the difference between trying to date someone by asking their ideas of family and what they like to do for fun versus rigging tiny cameras throughout their house, car, and office, and then analyzing thousands of hours of footage. Don't be that guy. Companies are all over exa-scale data now. Every Jurassic, physically-based business from airlines to health care is jumping all over the finely-grained know how. Their margins may be small, but their wallets are big, so they can pay a pretty premium to get their data act together and squeeze a few trillion pennies out of a crusty operation. Funny how people typing money into existence in New York -- simple data at large numbers, are fueling an over-heated economy that's getting pathological about taking complex data to reap relatively smaller numbers. Maybe that system could have used a good a priori model, like 300 years ago, and somebody could have pointed out the asymptote of desperate assimilation that ultimately swallows it up into a Godelian Black Hole. But I guess by then it'd be about time for Christmas, or Solstice if you're more of a pagan bent. Shhh, it's snowing data. Stick your tongue out and catch some. [You can also read the rest of Dugan's series about Metal Gear Solid 2 predicting Facebook, the industry's Wall Street envy, the joy of vector meme, and peripheral visions and dreams.]