12 MIN READ
Analysis: 'Socially Awkward' - It's Beginning To Look A Lot Like Metrics
Continuing his series on social games, game designer Patrick Dugan explains why metrics are both are "the Promethean firebrand that sears open the chains of blind intuition", and "a trap!"
I was that guy on the left, before I died. Well I'm alive again, and I'm here to tell you, undecidable propositions are still inevitably produced in any formal system with a finite number of axioms. So get over all your earnest glow about this metrics empire we've built up around us, or you will alienate your closest allies. As Admiral Ackbar would say: "It's a trap!" The numbers are making us number. Number and number and number. Around and around it goes, clicking 1.4 times a second for 87 clicks per session of an average 2.4 sessions a day over 450 thousand daily users... USERS!!!!!!!!!!!!!!! Get ready to take your next hit, soccer moms! The confirmation bias strikes again. Every one is trying to see what they already wanted to see, and then, pop, whaddya know? They see it! I'm both split and tested on the matter. It's all about finding that point of pricing where you get the greatest net revenues; if you price higher you'll get fewer people but more money per person -- at what price do you reach a diminishing return? That's become a science. Early game flow has also become something of a science, with each consecutive click measured, showing where you lose people. If you're losing people, change it, maybe you get more people committed. There are statistical principles at work that demand certain design choices. Godel's Theorem proves that whatever matrix of numbers you throw up, there will arrive a crease in logic, a rupture of your frame of reference, where an undecidable proposition will be produced, telling you neither true nor false. The way that kind of thing tends to be digested in the corporate world is simple: it isn't. Then the apparatus reaches its point of perfect adaptation, to live for a quarter or two in glory, and then declines before being put to sleep. Zynga and those who model themselves on its example are like the Nuclear Power industry. They're really popular these days right? Ah, it's a cheap shot, but I'm not being paid by the word so let's take it. They process data and leverage chain reactions in clusters of high energy people particles. The data is the uranium, the game loop is the reactor, the BB-summary-write-query-visualization-crossRef infrastructure is the reactor antechamber where energy is converted into a usable resource. Most of the time, it's glorious, like 2 Giga-Watts of capacity is glorious, each million DAU a golden goose lined up in cages. But sooner or later your users' behavioral probability distribution will go kurtotic, the reactor will melt-down, and your goose will be, well, cooked. Here's another metaphor for you, since we're getting to the part of the essay where I have to follow up with something constructive. Let's say you were a scientist, a behavioral psychologist maybe, serious enough not to think B. F. Skinner is an odd name. You set up an array of sensors in a rat maze. So far, so good. You run millions of rats through, you know exactly how many pellets they take, how often, with how much navigational difficulty. But you want more, you want to be able to statistically model their behavior and know with a high degree of certainty what path a rat will take. Someone I know actually had this problem with a social game; they wanted to amplify the resolution of their model to a predictive extent. I said: "You're trying to solve the wrong problem, you're looking too closely and not at the big picture. Instead of trying to finely measure a maze, give the people a garden and figure out how to model it with a decent enough resolution that you have the same Business Intelligence capability that you have now with these stupid fucking rat maze games!" When your data gets non-linear, you need to innovate with the back-end architecture to be able to make meaningful turn-around times on analysis. Zynga uses Vertica to crunch graph analysis and identify the highest impact USERS as they spam other USERS into USING. I know this because it was in a white paper. That's cool, that probably helps with retargeting, getting the cloud of people to pulse like a vibrant plasma as active USERS USE each other. Every successful social game has operated with a brittle nozzle of USER-generated content and charged the exploration of that content sea with incentives to drive retargeting - high resolution graph analysis can help you do that. But what if you turn the nozzle back just a little bit more? Will your apparatus hold? All the really good BI people I met don't really care that much about BI, they're more interested in crazy shit like abstract algebra or Hopf maps or Bayesian networks. It's going to be a really fun field, because we're going to have to figure out how to answer questions about the rate, tenor, contour, multiplayer orientation and difficulty level of player-created content beyond weakly interactive decorative sprites arrayed on a 10x12 grid. Skill is one domain of player-created content, if you really think about it. A highly variable output with mathematically defined limits; sounds like a feast to me! Now I'm bringing the most new-school game design stuff full circle back to the old stuff, but anyone who has played a lot of Spelunky already knows what I'm talking about. Modeling skill-based performance in the context of social game economics is a challenge I'm approaching on a project. Basically I'm going to try and define a probability outcome and take an average, knowing that if the process must be repeated hundreds of times the average will be economically meaningful. Not sure how else to do it, would love for someone to tell me. It's easy to have Znyga envy. After all, none other than Justin Timberlake has informed us that a million dollars is not cool, and I guess a billion isn't even that impressive; now you need to shoot for $20bb and a dog. But I say: fuck Justin Timberlake. Seriously, fuck that guy. It's time to bring gameplay back. If you model your game from the bottom up, a priori, you can produce values that are reasonably accurate global maxima, rather than extremely accurate local maxima, at like, a thousandth of the expense of licensing Vertica and building a state-of-the-art BI department. It's also possible to model things that can't be deterministically derived by rendering them as constants, things like how frequently people would do something, how well they perform in a contest of skill -- these things allow you to ask very specific questions that are much cheaper to answer than general questions from the top down. Data can be sliced a billion different ways, correlations can be found in anything if you split the hay straws finely enough, but if you need very specific numbers it's more like finding a needle in a haystack with a metal detector. Or, if you prefer a more social analogy, it's the difference between trying to date someone by asking their ideas of family and what they like to do for fun versus rigging tiny cameras throughout their house, car, and office, and then analyzing thousands of hours of footage. Don't be that guy. Companies are all over exa-scale data now. Every Jurassic, physically-based business from airlines to health care is jumping all over the finely-grained know how. Their margins may be small, but their wallets are big, so they can pay a pretty premium to get their data act together and squeeze a few trillion pennies out of a crusty operation. Funny how people typing money into existence in New York -- simple data at large numbers, are fueling an over-heated economy that's getting pathological about taking complex data to reap relatively smaller numbers. Maybe that system could have used a good a priori model, like 300 years ago, and somebody could have pointed out the asymptote of desperate assimilation that ultimately swallows it up into a Godelian Black Hole. But I guess by then it'd be about time for Christmas, or Solstice if you're more of a pagan bent. Shhh, it's snowing data. Stick your tongue out and catch some. [You can also read the rest of Dugan's series about Metal Gear Solid 2 predicting Facebook, the industry's Wall Street envy, the joy of vector meme, and peripheral visions and dreams.]