Game developers must reassess outdated production processes and mature their business models -- true globalization goes beyond outsourcing, and here are the risks, realities and rewards.
It's time for game developers to reassess their outdated production processes, grow up, and mature their business models. It is time for us to admit that most companies' current production models are not sustainable or are completely and totally broken.
Making games is getting harder and harder each year -- as well as exponentially more expensive. For most teams, the old production models aren't working, and most teams and companies are struggling to figure out how to re-invent themselves and stay in business. The days of needing 50 to 200+ developers working for two to three years on a game must change in order for us to find a way to become more profitable.
Like many other technology and manufacturing industries have done over the last century, game developers must learn to change and adapt their business and production models in order to survive. Over the last few years there have been a lot of companies turning toward outsourcing as a way to save money.
However, most teams just see outsourcing as a way to make cheap art. Production Globalization is about taking outsourcing to a much deeper level. It includes being able to work with teams across the world in much better ways than you ever have before. It is about minimizing risks in production and not creating more.
Production Globalization isn't just about making games cheaper, but about restructuring your companies, teams, and methodologies to be more flexible as needed and to allow you to use the best possible talent for the job when you need them and to not have to continue to pay them when they aren't needed. It is about making your teams much more scalable and adaptable to the projects needs from day to day and month to month, and not having to keep a large internal team around which is only productive part of the year.
This article will show you how to build your team, your infrastructure, culture, production processes and technologies to maximize its global production capabilities. You will also learn how to deal with cultural issues, time differences, differences in process, technology issues, approvals and other issues which plague globalized teams.
Production Globalization is the process of adapting your production processes to allow you to work with teams from around the world. This means that you are in essence creating a "virtual team" of people who aren't located in the same facility, but who must work together daily (or at least very regularly) to be successful.
This not only includes being able to work with teams in China, but also teams down the street. Production Globalization is the process of adapting your game development tools and pipelines to allow your team to hire other teams and individuals to help you develop your games. Production Globalization is not new, and many teams use it to some degree, but very few have effectively formalized it and optimized their teams and cultures for it.
Production Globalization is not just "outsourcing" parts of your game. Outsourcing is generally thought of as having another team help you with the artwork for your project, but is rarely applied to other disciplines. The term "outsourcing" could be the same as Production Globalization, but it has many negative connotations already, as most stereotype it as referring to just creating artwork on the cheap.
Production Globalization is most relevant to larger teams and projects, but most of the same principles will apply whether you have five, 50, or 500 people on your team.
"Work for Hire" is generally hiring another team to help you with some aspect of your project. This could be with porting some version of your project to another platform, or doing some aspect of the project which your team is unable to do, like the multi-player.
"Contractors" are generally individuals or companies who come in to do one thing on a project, like create storyboards, write dialog, or create some rendered cutscenes.
For many teams, however, the thought of using another team to do anything but create some art or do a port is a scary thought and often not even something they would ever consider. So, when evaluating whether your team could benefit from changing your development process to a more diversified model, you really have to approach it with an open mind, and realize that it will take a little time, technology, and training to get right, as well as probably a major cultural shift at your company.
It is important to realize that Production Globalization can take place on a variety of different scales. It should allow you to utilize virtual teams or consultants from around the entire world or from down the street just as easily. It should allow larger publishers and companies to have flexible resources across many studios, groups and teams which can also learn to share resources in new and much more beneficial ways.
Every team, company and project will have different reasons to want to globalize their production model. Some of the reasons to consider globalizing your production pipeline include:
Reduce and control costs.
Use the best talent for the job.
Reduce hiring and internal staffing costs.
Not have to retain talent in between projects.
Be able to bring on more talent for a short period to meet deadlines.
Ability to get more work done per day (an overseas team is almost like having a second shift).
And many more are possible...
There is a perception with a lot of teams that any work done by someone else is inferior. So, some may argue that outsourcing really doesn't save you money. This can easily be true for projects and teams which don't understand how to manage and direct their outsourcing partners, or for teams who have chosen bad partners. For most people, however, outsourcing even their artwork is a positive experience and a huge cost savings overall.
Globalization isn't just about saving money, even though it can save you a lot.
Many overseas teams charge less than $200 a man day, whereas a top contractor in the U.S. can charge more than $1000 a day. Not hiring someone full time also saves you other costs -- such as not having to have equipment for them, space, and benefits.
However, there is usually a huge difference in skills which also must be evaluated. For example, I have worked with many concepts artists in the game and film industry, including some world-famous artists like Syd Mead, Roger Dean, and Craig Mullins. Yes, these industry legends are very expensive, but most of the time they are worth every penny.
They are super good, very fast, very professional, highly creative, reliable, and easy to work with. An unknown artist who is more junior is usually slower, takes more revisions, and doesn't produce as high of quality in their final draft - but there are always exceptions.
So, for each project and need, just like when hiring someone full time, you need to access all aspects of them and how it will affect your project, budget and schedule, and then make a qualified decision so that you will understand the true cost and benefits of globalization.
There is an amazing amount of game development talent around the world.
Developing video games is a relatively new field. Even playing video games in many countries is relatively new. Countries like China, for example, still don't have the Microsoft Xbox 360, Sony PlayStation 3 or Nintendo Wii available to buy, even though they have been widely available for years.
Console game playing outside of the U.S., Europe, and Japan may have been very rare or even impossible for many people, including the people now developing games in those countries. I myself take for granted that I have been playing computer and video games for close to 30 years, whereas some designers I know in other countries have not only been making games for only a few years, they may have only been playing games for a very short time as well.
Not having as much experience in playing games isn't a problem for some developers, but is especially difficult for game designers, who often use the comparison of other games as part of their "language" which they use to relate ideas. This is often critical, especially for more theoretical design concepts and storytelling.
So, when you look at the reality of games across the world, as well as different cultural issues, difference in education, difference in experience levels both in game and software production, it is easier to understand that not everyone around the world has the same abilities. This is even true among the different companies in the U.S., let alone in other countries.
So, it is important to evaluate each team, test them, vet them and help train them to do what you want. You can't assume that just because you hire a "professional" team or company, which are truly up to the task, even if their resume looks like it. You need to go into any new relationships with open eyes, honesty and a little paranoid cynicism to be safe.
In the end, the reality is that some teams are highly creative and some aren't. Some are really great engineers and some are hacks. Some are great artists, and some are just great at reproducing others' art. You need to understand what you need, what you are capable of providing, and what they team can truly produce versus what they claim they can do. You also have to know what is really important to you and what is just content and not really critical.
Some examples of team models which are starting to use a more globalized production model include:
Many companies have partnered with overseas companies, open studios overseas, or set up shops in lower cost areas in order to do content production in places which are cheaper. This is most often seen as straightforward art outsourcing, where the bulk of the game is still done by a single team, with just content being outsourced.
A few U.S. teams have adopted a more virtual model, where their small internal team is just the top leads and managers who come up with the ideas and then hire others to implement them.
Several overseas companies have recently setup offices in the U.S. and hired small creative teams in the U.S. The U.S. teams are driving the design, managing the project and more. This is often done because of their lack of experience and lack of creatives who are able to design more original concepts.
There are many other industries which use a diverse production model where many people from all over the world and at different companies work together to make a much better product than they could do on their own.
The Computer Model
At most levels, game development and software development in general is no different than manufacturing a car, electronic device or many other complex items which have a lot of parts. A computer is something which has a myriad of complex components which must all work together seamlessly and be the best possible all around in order to be competitive.
If one or several components are inferior, it can either frustrate consumers so that they don't buy your product, or it can lead to a variety of other unforeseen issues. If you go and buy an Apple Computer, you are likely to get a CPU made by Intel, a hard drive by Fujitsu, a screen made by Samsung, memory by Hynix, and so on.
A single manufacturer cannot design and build an entire computer on its own. Apple learned this the hard way. But now, Apple can use some custom and some off-the-shelf solutions to find the best compromise of price, performance and features to make an incredible product.
The Hollywood Model
Hollywood has had its own production model for years -- it's both very similar to, and yet impossibly different than, game development at the same time. In Hollywood, most blockbuster movies are funded by a major studio (similar to the large game publishers like Activision). The studio is responsible for vetting the idea, helping make it better (having the script re-written) and then greenlighting it for full production (just like in games).
However, in most cases, the company who creates the movie is often a separate corporation which is established just for the creation of that one movie. This company is run by a senior executive team, which are usually managed by the creative and business leaders of the project. These leaders either then hire people (who usually work for unions as contractors) just for the one "show".
Then, the production studio will hire any number of different companies from all around the world to do many different aspects of the movie. You might have part of a movie filmed in several different countries, with maybe only a director or producer even being involved and on location for both. You might have dozens of different companies doing special effects, props and other aspects of the movie. In the end, it all comes together seamlessly and cohesively.
Aircraft Construction
Another analogy you can use is one of building an aircraft. Yes, a single company can employ everyone needed to build a plane, but they usually don't. Every aspect of building a plane requires a different specialist, and few people can do it all.
You need people to design the plane, design the frame, weld it, do the electrical, engine, painting, interior work, windows, heating, maybe weapons and much more. Each of these jobs has specialized parts and tools it needs, as well as specialists who know how to do it well. A plane construction process is generally run by a large general contractor (like Boeing), who finds all of the sub-contractors to do the work on the plane and ensures it comes together correctly and optimally.
Video Game Production
A video game is really no different to develop than these other industries, but we seem to treat it like it is highly unique, and therefore because of that we can do whatever we feel like. The game industry is beginning to learn that making games is hard, and that the process involved in making games is now often as important as or more important than the ideas themselves.
Game developers, like these other more mature industries, need to learn that distributing their workload and partnering with other companies to create a better product is not only okay to do, but actually the best possible solution.
Many people that I talk to have had a major problem with outsourcing their artwork. They usually find or claim that the quality just isn't the same, and that the hassles with outsourcing chew up any cost savings. A lot of this is due to their teams ultimately causing the problems, even though they won't admit it or don't realize it.
For example, if you are doing a Create-a-Player (CAP) system, and you want a team to create some shoes for you, don't get upset when you tell them to make you 10 pairs of shoes, and they deliver 10 shoes that you didn't want. With any team or person you hire, you either need to give them explicit instructions, be willing to pay for revisions or live with what you get.
If you want 10 specific pairs of shoes, send them pictures. If you want the shoes perfectly photo-real, send them the 10 pairs you want modeled and not some lousy, fuzzy image from only one angle. Their quality will usually only be as good as what you ask for. And, before you turn them loose on 10 pairs of shoes, make sure they do one to perfection, get your signoff, and then do the rest. So, in this case, your quality is only going to be as good as the source material and directions given.
So, what are some common problems to look out for?
The management team changing their minds.
Inability to make decisions.
Unavailability to review material and comment on it.
Unclear feedback and instructions.
Lack of documentation and relevant information.
Constantly changing specifications.
Asking for too many revisions or changes.
Changing technology, tools or pipelines.
Poor development tools and process which doesn't scale to larger teams.
Unrealistic schedules or delays in the schedules which affect the other teams.
Lack of communication.
No central team or project leader and point of contact who is "managing" them.
Differences of opinions as to how things should work.
Creative's who will change things because "they know better".
And possibly many others...
In the end, there is compromise to having an extremely experienced management team who can understand and leverage the differences between the global talent, along with a well established and documented production process and pipelines.
If your management team hires the wrong people or teams, makes bad decisions, is inexperienced, or keeps interfering with the production in detrimental ways, your project will be doomed to fail. If you have no process or pipelines, your products will suffer tremendously.
Globalizing your production process isn't for everyone. It is important that you understand and can manage the risks to the project.
Here is a checklist of SOME of the questions you should ask yourself before committing:
Do we have enough time to have someone else help us?
Do we have fast enough internet access to transfer builds and assets regularly? And will they?
Do enough of the teams speak the same language?
Can you be in the office during the day at all during the same times to have virtual meetings?
Do we have remote server access, like VPN?
Do we have source control that an outside company can access?
Do we have a regular build process?
Do we have documentation, both for our game design, technologies and tools?
Do we use any proprietary technologies we can't let a third party use or access?
Does the third party have experience doing what we need them to do?
Will any personality issues arise between the internal team and third parties?
Will the other team members delivery schedules allow this?
Having external teams isn't for everyone. However, especially for larger projects, it makes a lot of sense. But, I have still seen even small teams doing casual games virtually, and being successful at it.
One of the reasons that Hollywood has been successful in setting themselves up for a globalized production model is that they have invested a lot of time and resources to build up their infrastructure. They are investing in not only elaborate video conferencing solutions, but a wide variety of other technologies to help people share their work more easily.
For example, one of the major film studios in Los Angeles has set up several major teleconference rooms. Instead of just seeing everyone in a little window on a TV, they actually built identical rooms at both locations, and set up screens that are almost the size of the room, so that it looks like you are sitting on one side of the table in a huge room and they are sitting on the other side.
Everyone you see is lit the same as in your room, and is the same size as they would normally be. They made it easy to share digital files back and forth between the same rooms and have built-in tablets so that everyone can draw on the screens they are looking at and comment on them. They even set up integrated fax machines that let you easily "hand" a piece of paper to someone in the other room by putting it into the fax on your table -- and it pops out on the other table instantly.
The Cisco Telepresence Room is an example of how the video conferencing room at the movie studios is set up.
These same studios have set up amazing project tracking software systems which can allow them to see and manage everything in a project. Anyone can quickly and easily pull up any asset on a project, see every change, who made it, who requested it, how much it cost and more. A director can instantly pull up any shot and make notes and comments on it, ask questions and much more. Their system also manages many terabytes of data efficiently.
With each studio sometimes generating several terabytes of data each day, they had to find new ways to manage it, store it, find what they wanted, move it around the world and keep track of it. The studios also needed to significantly improve their tools to allow sometimes hundreds of people to work on the same shot in real-time.
So, while this may seem excessive, the Hollywood teams are successfully managing large teams and budgets around the world using these systems. They don't have these fancy systems in place to just look fancy, but because they have been proven to make the Globalized Production model work far more efficiently.
Even just a few years ago, thinking that the game industry would ever need the sophistication of a large Hollywood studio was hard to imagine. But, for larger multiplatform game production, MMO production and the development of other major titles, the game industry now needs to mature and realize that we need "tools" to work better in large teams especially.
If you have lots of extra space at your office, another example of globalization can include creating a partnership with another company for the project, and having them move into your office temporarily to work on the project. This practice is very common in larger cities like Los Angeles, where there are lots of teams near each other. Co-location can be a major help to both companies.
If you expect to have a VERY successful virtual team, especially one which is truly global, then you need to be willing to invest in some tools and technologies to help you.
Scheduling is tough enough to do with a local team, but even worse with an international team. Finding a good scheduling tool, especially one which works online, can be a lifesaver. Unfortunately, there are no really great solutions out there.
Technology is critical to standardize and follow critical production standards with. Having great secure source control, build systems, merge processes, asset tracking, version control, a componentized engine structure and other technologies standards is critical.
Asset Management needs to have a robust solution, and not just checking in assets to source control. You need to make sure you can review, track, give feedback and approve all assets.
Documentation also needs to be robust, and more than just a large GDD or other series of documents thrown on a server somewhere. This is part of the total communication problem, in that everyone on the team needs to not only have great documentation, but be able to search it easily, find the things they need, know when changes have been made and so on.
Communication is critical to your success. Dealing with virtual teams and doing conference calls is still harder than in seems, or more expensive than it sounds, especially to some countries. Making regular calls to China is expensive; however Skype, Ventrillo, and other VOIP solutions tend to break up badly when going to Asia or if too many people (maybe more than five or 10) are using it. Using those solutions in the U.S. or some countries can be fine, but don't assume they will work in Asia and some other parts of the world. This obviously rules out great international group teleconferencing for awhile. So, finding a good teleconference and video conference solution will be important. Fortunately there are some great advances in these areas right now.
Approvals are also extremely important to track. You need to know where everything is at, what needs to be reviewed, where everything is at in the pipeline and so on. Otherwise you will risk a very upset developer and a bad relation which can lead to other problems. Having a software solution to manage approvals is critical, even if it is just an excel file which is manually tracked.
Depending on the teams, your tools, pipelines and projects, you may also need to invest in better source control, build processes, and other technologies to help the teams work better virtually.
Dealing with the difference in time zones can also be tricky. You need to evaluate where your global teams are based and how your team can manage them.
For example:
9 am in England is 3 pm in China.
9 am on the U.S. East Coast, however, is 8 pm in Japan.
9 am on the Los Angeles is 9 pm in Moscow.
9 am in Chicago is 11 pm in Australia.
So, you need to understand if you team is able to deal with the differences in time zones. Some solutions are that your team, or some from your team may need to work longer hours, shift their schedules, or find a technology solution to help them communicate better if they are rarely able to talk on the phone or video conference.
One of the things which many teams and managers fail to properly understand is the differences in cultures between different teams. There are cultural difference due to society and the country you are in, as well as company and management cultures to overcome. Often, the cultures at other companies get in the way, even though people never realize it and never think to ask about it.
Some of these problems are just human nature as well, which must often be overcome in a remote environment. Many of the cultural "issues" are unwritten and not something most will admit or talk about. Some of the "issues" may also only occur with specific people, those in certain age ranges, and so on.
For example, when people don't understand you exactly, because they don't speak your language well, they will often keep saying "yes," even though they really didn't get it. They're often either too polite to make you repeat yourself, or too embarrassed. This is why I always follow-up oral communication with a written task list and explanation, just to make sure they really understood me, and didn't forget something.
In some cultures (especially in Asia), it is often considered rude to ask questions during or after someone's lecture. They are "taught" that to ask a question, especially in public, is to imply that the person gave a poor talk, and therefore did a bad job, which is considered insulting. So, often in a group environment, they will not ask questions. Therefore, it is often necessary to make sure you have contact with the team members directly.
Also, in a related problem, people often are afraid to ask questions, because they think it will reflect badly on them. Or, they may not want their boss to know they don't know, or may not want to bother them because they are busy, so the questions go unanswered. So, communication and lack thereof is a major problem for all teams, even within the same company and building, let alone globally.
Language is a major cultural issue. Even when working in England, I found that I often used words which meant different things in American English versus British English. On more than one occasion, this led to some confusion and awkward moments. This problem can be compounded exponentially when your common language you are working in is not native for one or both of you.
We often use popular culture to explain things, especially other games. However, we assume that people have watched the same shows, played the same games and so on. We also usually assume that they had the same experience as us. We have to be careful when assuming that people know the same things that we do. We also can't forget that some shows are even changed for international audiences, such that a character may be changed for another culture, to make them more acceptable.
Storytelling is one place where most cultures and writers in different countries are often on completely different pages. For example, what an American player likes is usually opposite what a Chinese player finds fun and interesting. Sometimes these differences are cultural, and sometimes it's just differences in taste. A lot of games which are successful in places like China don't have the same story in China and the rest of the world.
Even just doing business overseas can be shocking for many people. A lot of business, especially in Asia, is done at strip clubs and other strange locations that seem unprofessional to some. Their cultures may have other unwritten rules about what etiquette to use while having meals, going out drinking and things like that. So, especially if you are visiting a development partner, it is good to get advice if you are new to doing business there and think that something may bother or offend you or them and ruin the relationship.
There are many other cultural issues to be aware of when dealing with teams in other countries. It is good to talk to some people and try to not be shocked when something weird or different happens and just try to adapt to it.
Many engines, including many popular middleware engines, are not structured in a modular or componentized structure, making it really hard to work on a single system or part of the game in isolation. In order for really great Production Globalization to work, it will probably take some minor to major changes in your technology and engine structuring.
However, once accomplished, and once you have your technology modularized, it will significantly help everyone on the team, whether you are globalizing your production or not. The point being is that it is critical to your long term success if you develop your technologies in a way which lets many different people work on parts of it, without needing access to or even knowledge of the entire engine and game.
Longer term, you will also come to realize that we also need to move away from being a technology driven industry, and become much more of a content driven industry. In the film industry, if you talk to the leading technical directors at major companies like Pixar, ILM, and Dreamworks, you will find that not only do they all share code, but they get together and meet regularly on how to standardize their technologies.
They don't want to have to re-invent their lighting model for each project, let alone re-invent the cameras too. In games, we like to re-invent things far too much, and must begin to move away from that in many cases.
Now, understandably for games, unlike films, having a technology advantage over your competition can be a huge advantage, so sharing every core algorithm with each other isn't feasible. However, this doesn't mean that tools and core concepts can't be shared, while maintaining some competitive advantages.
We also can't underestimate how critical it is to standardize your toolset. Unreal Editor 3 is very popular, not just because it is good, but because many designers know it. Being able to hire designers and others and have them get up to speed quickly saves everyone time and money in the long run. Artists have been mostly able to standardize on 3DSMax and Maya, which grown more similar each year, but designers and programmers still need to continually relearn everything half the time (or most of the time) they switch projects or companies, which hurts everyone.
This also means that we need to be okay with reusing our technologies. We've already seen a lot more reliance on middleware, which can be a very positive trend. However, I know many companies that allow their internal teams to use any technologies they like, and even change their technologies significantly from one project to the next, even when doing what should be a simple sequel.
So, until we can break from some of our technical bad habits, it will still significantly hold us back at many levels in many of our projects. It will be impossible to quickly bring on external designers and programmers, unless you use some kind of industry standard.
In order to understand how to globalize your production process, you need to:
Honestly evaluate all aspects of your company.
Understand what resources internally and externally you have access to.
Understand the risks of different production models.
So now that you have an idea of what Globalizing your team is about, and why it is important, you're probably asking "How do I do it?" In future articles I will get into more details on exactly this, but for now, you need evaluate your team and project and create a plan for them including:
Evaluate which team members are used during which part of the development process and for what capacity and decide which roles aren't needed regularly enough.
Determine where your current team's weaknesses are.
Build a plan for modularizing your technology, and improving your technology process to ensure that outside teams can utilize it.
Evaluate teams worldwide and begin to build relationships with other companies who you might be able to outsource to, and ultimately who you trust.
Determine what parts of your development process work, and which could be improved -- and then determine if they can be improved by using outside help.
Analyze the risks in production pipeline, and create a risk analysis for them, weighing what options you have to minimize the risks (and cost) associated with each one.
Depending on how much you want to change your process, these may only be the first steps in understanding how to improve your process.
No matter what you think about the idea of adapting globalization within your production process, hopefully you are being smart and at least now evaluating how you are working, and how it is working for you.
Globalization may not be for everyone, but for most typical game developers it could mean the difference of staying in business or going under do to cost overruns and other issues during your game development.
No matter what you do however, the most important thing is to be honest with yourself and your team about what really needs to happen, and then create a plan to try and make it happen over time. The industry can't shift its paradigms overnight, but this change will happen, one day soon. Will you be ready?
Read more about:
FeaturesYou May Also Like
