When you think about the big levers of revenue and profitability in online games, a lot of things come to mind:
- Cost-effective customer acquisition through great marketing programs and viral word of mouth
- Delivering new features that improve player engagement and stickiness metrics as well as LTV
- Making systematic improvements through cohort analysis and A/B testing
What doesn’t necessarily come to mind is DevOps, a software development method that stresses communication, collaboration and integration of developers and IT. But done right, an investment in DevOps function can actually deliver more to the bottom line than the best-laid marketing campaign. Most importantly, DevOps is your first line of defense for your brand and health of your subscription revenue. Here’s why.
The Full Cost of a Technical Issue Outlasts the Problem Itself
Free-to-play gamers are a fickle lot. If your game is offline when they want to play, they won’t come back. Lose their purchased goods; they won’t come back. “Our games consume our players’ valuable and limited entertainment time,” says Albert Ho, Executive Producer/Product Manager for Platform at Rumble Entertainment. “If anything isn’t behaving correctly in our game or if key features are running too slowly, we could lose that customer forever.” According to Ho, a bad player experience can reduce revenue from new players by as much as 70%.
Of course, understanding the magnitude of your technical issues assumes that you know about them. The “unknown unknowns” are a developer’s (and your CFO’s) worst nightmare. If you aren’t seeing and solving in the moment, your ship is slowly sinking.
Slow Play = No Play
Sluggish app performance is no better than outages. In fact, it is often worse as the root causes tend to be hard to find. No matter what the cause (and many causes of slow performance are out of your control), you lose. Players won’t come back, and LTV takes a big hit. That’s assuming you even know about the latency and issues.
Innovation is the Name of the Game
At the same time, players are always expecting a better experience. If you aren’t pushing code several times a week, you’re probably not keeping up with your competition. You need to be ready for the inevitable – that issues will crop up. And that someone will need to fix them right away.
DevOps is Where It All Comes Together
DevOps is the function that keeps your release cycles on track and ensures that the quality of player experience stays high even as your game evolves. By maintaining real-time visibility into the stability and health of your games and monitoring key performance indicators (KPIs) that serve as an alarm system, DevOps knows when when action is required and can galvanize the development team with objective data on the magnitude of a potential problem.
DevOps is Closer to the Customer than Marketing
While Marketing focuses on acquiring players for future revenue, its activities have a fractional percentage of impact on current players. A stellar acquisition campaign may convert high-single digits into paying players, but its success is dependent upon game performance, managed by DevOps.
The efforts that DevOps makes to deliver a quality experience have a virtually 100% impact on both current and future revenue – whether the team is preventing server outages, eliminating performance bottlenecks, or monitoring the stability of new features. With access to the right data, DevOps can tell where players are struggling—and even which players are struggling—so you can move quickly to protect Day 1 Retention, Day 7 Retention, and LTV.
Pound-for-pound, a dollar invested at the bottom of the funnel—in protecting your player experience—will generate more revenue than a dollar invested at the top of the funnel, in acquiring a new customer.
A Winning DevOps Approach Starts with the Right Mindset, the Right Data, and the Right Processes
You won’t find a single, perfect DevOps job description out there on the web. So rather than searching for one, we suggest that you take the advice of Jon Gifford, Loggly’s Chief Search Officer and architect:
“Your developers should want to know how their code is behaving in production – they should ‘know the shapes.’ Your Ops people should want to know about the internal monitoring, and should be comfortable using it to dig a little deeper than they otherwise could. There should be as few barriers as possible between the two groups. They have different specializations, but the end goal for both should be a smoothly running, high performance, well understood system.”
Instrument Your Games for Real-time Visibility
The not-so-secret ingredient to real-time visibility is log data: data on behavior within your own servers, the network, your hosting provider, a third-party service you use like payments or social networks. With the right instrumentation, you can understand what’s going on in all of these areas at that same level of granularity that you understand game play. In order to get the most out of your log data, you should:
- Structure your logs so they can be easily parsed and analyzed. JSON offers a huge amount flexibility in defining and updating your fields.
- Log unique session or user identifiers so you can trace events from client to server side, and across multiple servers and APIs. One of the unique challenges for game developers face is the huge diversity in players’ client hardware, software, and networking environments.
- Consider machines, not humans, as your logs’ end consumer. Engineers tend to push back on idea of adding instrumentation because they don’t want to see their pretty logs full of “clutter.” But a log management solution can sort through this clutter in the blink of an eye, as long as it’s structured.
Monitor KPIs that Signal Application Stability and Health
If you’re only learning about problems when your players complain, you’re putting as much as 70% of new revenue at risk every time a release gets pushed. You need to be monitoring KPIs on a regular basis, and especially after every code push. You might look at:
- Number of stack traces and error exceptions
- Incidence of client-side problems such as failure to connect, game lag, characters disappearing, or models not rendering
- Player activity in key features
- Completed and uncompleted transactions
And of course, once you discover an issue, DevOps can work with development to determine its cause and bring the game back to a healthy state.
Close the Loop with Customers
In the case of a serious problem, DevOps can play an important role in player recovery. After all, it’s less expensive to save a player than to replace one. DevOps is in a unique position to quantify the impact of a particular issue, determine its cause and effect, and provide data on exactly which players were affected. Then, Marketing, Customer Support, or Community Managers can proactively reach out to affected players and execute retention programs.
Make the Investment in DevOps
Investing in DevOps—and the instrumentation and tools that support DevOps activities—is a winning strategy for game developers. There are a host of technical issues that can—and will—undermine the player experience, even in the best-designed games. And by doing so, these issues are diluting your marketing spend. Conversely, delivering players a consistently great experience amplifies the impact of your investments in customer acquisition by ensuring that you will get the highest possible retention and LTV for each new player.