[Industry veteran Miller looks at the leading Agile methodology for game development, suggesting the ten top pitfalls - and ways to overcome them - for those using Scrum to manage a video game project.]

There has been some hype about using Scrum for game development, including presentations at Game Developers Conference. There are books and many internet resources that describe Scrum, and this article assumes the reader is familiar with Scrum and Agile methodologies for product development.

Scrum can be a beneficial for some kinds of software projects. There are, however some pitfalls that are easy to run into when employing Scrum to manage a video game project.

Some hazards can occur because the importance of well-established, existing best practices is ignored. There can be consequences when Scrum is used as a replacement for existing best practices. Here are 10 pitfalls that were experienced on a recent project while employing Scrum methodology.

10. A Game Design Document (GDD) isn't needed anymore because the backlog spreadsheet replaces it.

While maintaining both a GDD and a backlog, the team is drowning in an overwhelming amount of paperwork every day. The ScrumMaster or Producer responsible for the backlog ends up spending less time writing or maintaining the GDD and as a result there is a lack of product design specification. Team members are left guessing what code to write or what components or assets need to be constructed in order to make the game that is going to ship.

Lesson Learned: Backlog spreadsheets are not a replacement for printer-friendly game design documentation. Team collaboration is not a substitute for someone putting the product design specification in writing.

It's hard to see up front that the lack of printer friendliness of the backlog spreadsheet is actually going to inhibit everyone's ability to communicate. Before backlog spreadsheets, it was possible to print out the GDD and bring it to the team meeting and scribble notes on it. It was also easier to provide feedback in writing. By printing out the GDD and bringing it to the meeting, you could talk about the design and refer to it without sitting at a computer.

While using a backlog spreadsheet as a replacement for design documentation, one issue team members realized is that after a meeting they walk back to their desks, start to concentrate on tasks and have forgotten seven of the eight points that were discussed during the meeting. The solution proposed was that everyone needs a laptop to bring to the team meeting so team members can type into the spreadsheet simultaneously. Imagine the team meeting is like a LAN party and everyone can see each others' cursors on their own view of the spreadsheet.

Unfortunately spreadsheets do not have a multiplayer mode, and the company is not going to buy everyone a laptop just for Scrum methodology. The obvious solution is everyone brings a notepad and pen to the meeting and takes their own notes, but why didn't the designer put the design in writing in the first place? Why is the design done on the fly and everyone has to take notes like an executive dictating a letter to multiple secretaries at a time?

In multiple team project review meetings the ScrumMaster has remarked "We need more communication on our team." All the developers then roll their eyes and ask "where is the design specification in writing? Wasn't the printer-friendly GDD a form of communication?"

Best Practice: Before there was Scrum, writing the GDD in Microsoft Word and putting it in source control was a best practice. A wiki as the GDD is perhaps better than a Word document in source control.

9. Interrupt what people are doing to have them come to the daily 15 minute stand up meeting.

Schedule the daily 15 minute stand up meeting on everyone's Outlook calendar with a 15 minute reminder, and then the ScrumMaster shows up 2 minutes late and politely waits for those team members who show up 10 minutes late to the meeting.

Question: How does a 6 month project get a whole year behind schedule? Even if you haven't read the book Mythical Man Month, it is common knowledge that the answer is one day at a time. The 15 minute daily huddles are not more important than contiguous hours of concentration and in-the-zone productivity.

Interrupting programmers, a.k.a. "committed pigs" of Scrum, in the middle of something technical, just to make them come to the team room only to ask them "Are you blocked?" is a good way to derail a half day's worth of work as well as derail the entire project. Every day the conversation might go like this:

ScrumMaster: "Are you blocked?"

Engineer: "No, we were just working on making the widgets slide across the screen smoothly by using the first Hermite basis function to interpolate an 'S' shape curve."

ScrumMaster: "Since I'm a non-technical person why don't you say that in English?"

Engineer: "The UI will look cool with smooth animation."

ScrumMaster: "Do you have an estimate for that?"

Engineer: "Tomorrow."

Lesson Learned: The 15 minute daily stand up meeting was supposed to ensure that team members do not spend a day getting sidetracked by tasks that are not important to shipping the product. What if, before there was Scrum, developers had already been through a couple development cycles on various projects and had an established track record of working on the correct tasks and shipping products on time? How does the 15 minute daily meeting correct something that wasn't broken? It doesn't.

Lesson Learned: A 15 minute reminder for a meeting that starts 10 minutes late just wasted 25 minutes of time for everyone on the team. Suppose that after the meeting developers go back to their desks and spend an additional five to 15 minutes to concentrate and focus on what they were doing before the meeting. The time around a 15 minute meeting adds up to about 40 minutes of distractions during an eight hour work day.

This can amount to weeks of man hours in the course of the development cycle. The question here is: was this time well spent? Does this daily meeting accomplish what it is supposed to accomplish, and does the benefit outweigh the cost? Is there a better way to accomplish the same thing, and without consuming the time?

Having the backlog in an Excel spreadsheet separate from the bug database just makes more work for everyone to maintain a to do list in two separate tools. It's obvious that development tasks and defects both have the same kind of data. In other words, database software does not know the difference between a dev task and a defect.

The only difference between a dev task and a defect is that a dev task comes before implementation and a defect is after implementation and includes steps to reproduce it. There should be no reason why dev tasks and defects can not be tracked with the same database, project management tool.

Best Practice: Some development tasks take a couple minutes to complete and some development tasks can take multiple days to complete. Instead of having the same conversation everyday with a Scrum Master, it would be better to put all tasks in a task management database.

The task database needs to have some kind of folder or query mechanism to view which tasks are being worked on today. Each person on the team can have their own folders and the folders can be called "WorkComplete", "WorkInProgress" and "WorkToDoNext". Each team member updates their own folders as they are switching tasks, and when they claim complete on a development task or claim fixed on a bug.

Everyone on the team, from their own machine, can see everyone else's folders, so they can see what other team members are working on. The time estimates or time remaining can also be entered into each task item. If anyone wants to know what team members are doing today, that person can just look at the "WorkInProgress" folders. TestTrack, DevTrack and FogBugz are three commercial off-the-shelf defect-tracking and project-management database tools that have features that can support this.

If you find value in visualizing the Scrum task board or if you still use a Gant chart, consider making a little graphics program that takes everyone's folders and tasks from the database and renders the tasks as nodes (like task cards on the Scrum board) in a directed acyclic graph of task dependencies on a horizontal axis time-line (like a Gant chart).

Then you can use a projector to display the rending on a wall. Imagine you can add little progress-status bars on each task for the hours estimated and hours remaining (like health bars above avatars). You can even have animated effects for when team members are at their desks claiming complete on tasks (like the animation of files going into the recycle bin in Windows Explorer).

Imagine these animated effects will look like a fireworks particle system when you reach 100 developers on a team. Developers whose actual hours closely match estimated hours will receive +3 experience points and completing a cycle will allow them to level up. W00t! Executives will be so entertained to have the real-time visualization of teamwork and productivity that they will forget to play-test the latest build of the game.

Best of all, this electronic Scrum board will scale from small teams of 5 game developers to large teams of 250 game developers. Don't forget to implement smooth-zooming to hyperbolic fish-eye view for large graph visualization (see Google images for those words).

8. Moving all team members into a dedicated team room or team area does not cross-pollinate with cubicle assignments in the per-discipline-organized cubicle farm.

It's good to have engineers sit next to each other so they can work through solutions to problems. It's also good to have engineers sit next to content creators so the engineers can offer customer support for the custom authoring tools. It's somewhat of an inhibitor if a developer has to walk across the building multiple times per day to collaborate with another developer. Part of the problem here is that desks and cubicle walls are not moveable.

Lesson Learned: If you want collaboration to happen, don't mandate it first and then inhibit it at the same time. Instead first enable collaboration by removing the inhibitions, and then let collaboration happen by coincidence or out of necessity.

Best Practice: A way to make collaboration uninhibited is to first put desks on casters. Second, give everyone un-interruptible power supplies and wireless network connections. Next, consider getting rid of some of the cubicle walls. Team members are now enabled to unplug, and move their desks around the open bull-pen work area without shutting down and rebooting machines.

Thus a half day of face-to-face collaborative productivity is not inhibited by the hour it takes to move equipment, re-connect cables and reboot. Sure, un-interruptible power supplies for everyone are expensive. The hours of lost productivity, inhibited productivity, lost data, and damages from dropped equipment are also expensive.

Finally, tell developers it's urgent to get development tasks done on time and they can push their desks and computers anywhere they need to in order to get work done. The result is that developers will find it compelling to push their desks next to each other out of necessity to be productive, and not because of the mandate to fulfill the prophecy of buzzword project management.

For added comfort, each development team or development group can have a designated team area and moveable privacy screens and/or a team room. Make sure to provide big whiteboards on wheels too.

7. A team member says "I'm not doing something right now because I was told there needs to be a task entered into the backlog and the task needs to be assigned to me by the project manager."

Lesson Reinforced: The proactive person looks for the next task that needs doing, asks customers if everything is working, and thinks about what can be done now to make next month's to-do list lighter weight. The reactive person finished the work for the current two week sprint and believes they are not responsible for anything more until the next two week sprint when they are assigned more tasks.

Lesson Learned: Telling people that the whole team is focusing on only two weeks' worth of work at a time, and they can't work on tasks unless assigned, can make people reactive instead of proactive. It's important that all team members can see the big picture, and are encouraged to take responsibility for the components and details of the big picture without micromanagement.

Best Practice: Adding tasks to the Scrum spreadsheet feels like having to ask permission for things that have trivial granularity. Developers assume ownership and responsibility for the project. Developers have foresight about what work needs to be done next month. Developers are able to proactively work on tasks ahead of schedule as well as provide customer service to each other without needing a task assigned.

6. Start managing stories and sprints before all the people are on the team. Decide that your Scrum stories and sprints are 100% completed months before the project even has a QA tester.

Lesson Reinforced: Without QA in the first month of the project, you might as well admit that you are planning to have a death march and crunch time. It is common knowledge that adding QA to the project up front is not just important, it's mission critical.

Best Practice: Throughout the development cycle, have engineers and content creators fix bugs as they go, so that the total open bug count stays at a minimum. Have QA on the project at the beginning of the development cycle to report bugs and to verify and close bugs. This is important for reducing crunch time due to overwhelming number of bugs at the end of a development cycle.

Every day, each developer can look at the open defect count and their own current dev task to do list. If there are more open defects than dev tasks, consider the bug count too high and spend part of the day fixing the lowest hanging fruit before switching to the next dev task.

Even better: each person on the team is customer-friendly and asks their customers and teammates if there are any issues that are preventing them from fixing their own bugs. Recognize that some bugs don't get fixed because they have dependencies on other people fixing other bugs.

5. Scrum Pitfall: The 15-minute daily stand-up meeting degenerates into a daily go-around-the-table-and-each-person-give-a-status-report.

There are no action items from the daily meeting and the daily meeting doesn't actually help increase productivity. At the daily meeting, the Scrum Master makes some remark that team members or the team's actions are "not Agile".

Lesson Learned: There is no value or purpose in practicing Scrum just for Scrum's sake.

Lesson Reinforced: "Round table status report without action item" meetings are a waste of time. Certainly the cost of everyone else's time combined does not outweigh the benefit of saving the meeting inviter's time.

Best Practice: Maximizing everyone's 40 hour a week bandwidth pipe is more important than a daily 15 minute meeting. If one person on the team has the lion's share of work to do and others are waiting for the next daily build or the next two week sprint there is something wrong with the project or the project management. The workload needs to be evenly distributed.

Best Practice: Make sure all meetings have an agenda, and the agenda is part of the meeting invite. An action item includes the task that will be performed, the name of the person who will do the task and should also have a deadline or at least a time frame. For example: "Joe will have placeholders for the character model checked into source control by the end of the day tomorrow, so designers and engineers can work with it this week."

If there were no action items for anyone after the meeting, there was no purpose to having developers stop working and gather into a conference room. Status reports and other communication can be done over email, instant message or a phone call.

4. Make Scrum mandatory.

Stakeholders and executives make Scrum mandatory for the development team and expect quantitative project completion predictions. Managers don't ever provide feedback to the team members, or reveal what value was actually gained from all those Scrum sizing exercises where developers choose numbers from a deck of cards.

Even worse: executives don't gain any value at all from the quantitative predictions that they didn't already have with the project manager's gut feeling. Or maybe the executives don't understand that the team members are spending time doing ridiculous number guessing exercises to size a Scrum story in order to obtain the quantitative prediction that no one looks at.

Lesson Learned: Communication needs to go both directions, not just one direction. That means up the chain of command and also down. Feedback from, and visibility of stakeholders and executives is equally important as providing those stakeholders with a summary of the project status.

Best Practice: If a milestone is completed on time or if a game ships on time, stakeholders, senior management, and even executives walk around the cubicles and say to individual team members at the bottom of the org chart some things like "Hey, I heard your team completed its milestone on time and your project is on schedule, great job! That's exactly what we like at this company! That's excellent work!"

The positive reinforcement from the top of the org chart is worth more than any gold that anyone can earn in any MMO. Note to management and executives: if you want productivity increased, make sure your positive words are more valuable and more gratifying to employees than their time spent shooting giant zombie spiders and leveling up.

Imagine if there is just one milestone that is on time and that on-time delivery gets personalized recognition and reward by both the VP of product development and also by that VP of marketing who never seems to come out of his/her 700 square foot corner office for anything. Compare that to the same on-time milestone when no one seems to notice or care about the hard work that went into it. Success breeds more success! Low morale breeds high-cost employee turnover.

3. Implement only two weeks' worth of work and write only the code based on what you currently know about the game design.

If you were not assigned to work on a task, then you can't work on it. Ignore any concern that the game design may change later on in the development cycle.

Lesson Reinforced: Bottom-up code design is not better than top-down code design. Scrum, Agile, and Extreme Programming cultivate and promote bottom-up code design. When there is lack of big picture design documentation, the tendency is to implement one thing at a time, and then modify and tweak according to customer's feedback.

Since there isn't any foresight about what code will get shipped and what code will be thrown away due to design changes there isn't a lot of care taken to make code maintainable. Adding features that were not planned for can be costly. Down the road band-aids and patches require time spent refactoring.

Best Practice: Make a game from low fidelity to high fidelity. The low fidelity version has prototypes of all the features. Code can be organized from top down and with understanding of all the features that are going into the product that is going to ship. Note that there is a difference between code and content. Perhaps the game content is what you want to be able to sprint and iterate on.

2. "Sprint Zero" is pre-production time.

Start the pre-production by putting all stories into the Scrum backlog spreadsheet. Pre-production only lasts two weeks. Ignore any concern that there still isn't concept art before Sprint 1.

Lesson Learned: "Sprint Zero" is not pre-production. Creativity and innovation cannot be managed. Team brainstorming might be visible while sitting in a conference room buzzed on energy drinks but sometimes the best ideas come to mind while stuck in traffic during the car ride home from work. You cannot demand that someone comes up with the best creative game idea instantaneously. It doesn't help to manage and track game ideas in a backlog spreadsheet, either.

Best Practice: Pre-production for any great product involves lots and lots of prototyping. Think on the magnitude of hundreds of prototypes. You would like to be able to mass produce these prototypes. Some ideas may never be more than a scribble on a little yellow piece of paper. Some prototypes might be full-featured software demos with hundreds of tuning knobs and ways to adjust content on the fly while the game is playing.

You need powerful prototyping tools that allow non-technical people to do lots of iterations every day. Pre-production does take time. The more prototypes that the team can create and demonstrate quickly will result in a better product.

1. Start managing the project by prioritizing tasks and asking for time estimates even before any GDD is written.

In every two-week sprint planning meeting, repeat the same behavior of prioritizing tasks before there is any design. This is about as dysfunctional as signing a production contract with a licensor before there is any design document or concept document. The worst case is then giving the licensor the freedom to make any change to the design at any time during full production with no regard for the cost of the team implementing the change.

Lesson Learned: The project management cart does not go before the game design horse.

Best Practice: Write a draft of the game design document and get feedback from engineers, content creators as well as the publisher and licensor. Do this before making a commitment to full production. Make sure that engineers and content creators understand the product design before you start prioritizing tasks and asking for time estimates.

Best Practice: You may have a mental checklist for sanity sake, and doing things in the correct order during a development cycle is important. Many developers have a gut feeling about death march projects. If your gut feeling says this is going to be bad unless the proper actions are taken, then it probably will be. Trust your gut feeling about game development over the Scrum consultant's advice. Scrum consultants don't make games; they get paid to give an all day Scrum Master certification training class.

Conclusion

Scrum and Agile methods are just tools in the project management toolbox. Suppose you had a crescent wrench, a vice grips, a channel wrench, and a pliers in your toolbox. If you just use the crescent wrench for everything without thinking about what it actually takes to get the project done, you are going to end up stripping a lot of your nuts.

During many game development projects, some lessons are learned the hard way. The hard lesson learned is that Scrum is not a silver bullet that makes video game product development more successful. There are many technical details, know-how and best practices that have been gained by years of experience developing games, one development cycle after another, analyzing what went right and what went wrong.

Employing Scrum as a complete replacement for already well-established process and methods that existed before there was Scrum is a sure way to make a project miserable punishment. When Scrum is added to the project to solve problems and ensure success, it can easily become the cause of more problems than it solves if the pitfalls are not remedied with appropriate timing.

This article has provided some contrasting insights about Scrum. Certainly before making the executive decision to mandate Scrum, it should help to hear from both camps: those who advocate Scrum religiously and those who approach cautiously and play devil's advocate.

---

Title photo by royskeane, used under Creative Commons license.
Additional photos by star5112, Tom Hughes-Croucher, and Paul Downey, used under Creative Commons license.