Cutscenes, in one form or another, have been a part of computer games from almost the beginning. There were the crude, pixelated animations of early arcade games, the live-action, bluescreen productions of PC games, and now the 3D animation of today's consoles. Purists like to say that cutscenes are a distraction, an unnecessary and annoying intrusion on the integrity of the game. Gameplay is king! And, even those who enjoy cutscenes would have to admit that there are times when they get too much of a good thing. Some games have so many cutscenes, or such long cutscenes, that you have to wonder whether the game designers had a movie in mind instead of a game. And then there are those games with such boring, confusing, ill-conceived cutscenes that it would have been better if they hadn't bothered making any cutscenes at all.
When you leave the interactive world of the game and enter the cinematic world of the cutscene, the rules change. You depart the continuous, spontaneous world of on-the-fly camerawork, and enter the world of traditional cinematic structure. As someone who has worked in both the game industry (at LucasArts) and in the film industry, (at Pixar) I look at cutscenes in games with a keen awareness of cinematic design. Shot compositions, staging, cutting, pacing: every aspect of what is seen in the cutscene has expressive importance and is there, exactly the way it is, because someone decided that they should be.
The rules by which these decisions are made are the rules of the language of film. Unfortunately, game cutscenes are often conceived in ignorance of the basic grammar of this language. And when you have a shaky grasp on the rules of grammar, it's hard to express yourself clearly.
There are some very beautiful cutscenes out there, and there are lots of bad ones. What are the problems that I see?
- Confusing, unnecessary cuts
- Hyperactive camera work
- Violations of basic rules of screen direction
- Shots that don't effectively express story points
- Indifference to lens choice
- Inattention to continuity
There are lots of ways that cutscenes can go wrong. What follows are four fundamental steps that can help in creating cutscenes that go right.
Step 1: Know Your Purpose
A cutscene such as this one from Tony Hawk Pro Skater 4 can define the beginning of a level.
The first step in making better cutscenes is being clear on what those purposes are.
- Advance the plot and give meaning to the dynamic progression of the game. The best games progress dynamically, meaning that as you play the game, the stakes get higher and the nature of your involvement becomes more intense. That's what happens in stories too. As the plot advances, the stakes get higher, until you reach the climax of the story when the ultimate issues are addressed. Mixing the dynamic of a story into the dynamic of a game gives another level of meaning to the gameplay action.
- Define the beginning and end of a game level. A cutscene at the beginning of a level sets the stage for the action that follows. At the end of a level, a cutscene signals to the player that he has achieved the objectives of that level.
- Give the player a reward. Cutscenes allow players to take a break from their efforts and just be entertained, so anytime a player completes a difficult task, a cutscene can act as a kind of reward.
- Introduce gameplay elements and provide the player with necessary clues. Providing a game player with the necessary clues and information that he needs to be sucessful is like providing a movie watcher with backstory information that makes the action of the story intelligible. The word "exposition" works in both contexts.
- Set the Mood. Creating fully realized, prerendered cutscenes does a lot to set the proper mood and immerse the player in your story. By doing so, the player gets a glimpse of what the imaginary world of the game might "actually" be like. When gameplay begins, the player will hold those scenes in his mind's eye and see the visually impoverished, low-poly world of the game through those high-res images.
- Define the mythology of the game. One of the things that attracts a player to a game is what I would call the "mythology" that surrounds it. That mythology is defined by the characters, the environments, the props and the overall look of the game. But, most of all, it's defined by the story. This mythology is something that game players will either buy into or not. It's one of the key elements that can make or break a product . Cutscenes are the place where that mythology is most forcefully expressed.
- Marketing, the secondary purpose of cutscenes. If a tree falls in the forest, and nobody hears it, does it make a sound? If a game company makes a great game, and nobody buys it, does it matter that its great? The sad fact is, you can make the greatest game in the world, but if the press and store buyers don't notice it, you're going to have a hard getting anybody to buy and play it. And, if nobody plays it, it might as well not exist. There are a lot of games out there, so, in the 10 or 15 seconds that you have at E3 to catch the attention of the tastemakers of the gaming world, you're going to want to show them something that will knock their socks off. Here's a news flash: video captured gameplay ain't gonna cut it. When it comes to promoting your game, there's nothing like a few minutes of slick cutscene action to create viewer excitement.
Step 2: Previsualize it
The second step in producing better cutscenes is to do better planning. In this case, another name for "planning" is "previsualization". To understand how previsualization fits into the production pipeline, it's worth taking a moment to look at how the whole animation production process works.
The Pixar Production Pipeline
Cutscenes are made the same way that any animation is made. It doesn't matter whether the animation is being produced for a game of a cartoon, there is a certain order that things generally need to go. As an example of a basic model of animation production, I'd like to outline the way things worked at Pixar when I worked there on A Bug's Life, Toy Story 2, and Monsters Inc.
- Script, Conceptual Design and Storyboard. In Animation, these three activities happen simultaneously. Unlike in live action film where you need to have a script before you can start storyboarding, with animation the storyboarding and conceptual design happen alongside script development as equal parts of the story creation process.
- Animatic. Once the storyboards begin to give shape to scenes, the Editorial Department begins to put them together in a story reel, or Animatic. I will talk more about Animatics in a moment.
- Modeling. After the story has evolved to the point where the company feels committed to actually making it into a movie, the modeling department begins creating all the characters and sets that have been developed by the Art Department in the conceptual designs.
- Voice. Voice actors are cast and the voice lines are recorded.
- Set Dressing. The Set Dressing Department is responsible for taking all the props and putting them together to form the sets in which each scene takes place.
- Layout. The is the stage at which each shot that was planned in the storyboards is finally put together in the 3D world with the characters and sets provided by the modeling department. I'll be talking in more detail about layout in a moment.
Notice that there are a lot of things that happen ahead of the animation step. To use a theatrical analogy, animation is the part of the process when the actors step onto the stage and perform. But before we have the performance, we need the rehearsals. In the animation pipeline, "previsualization" is the "rehearsal".
Previsualization is where you can answer all the critical filmmaking questions. How should the action be staged? Where should the camera be positioned? How should the shot be composed? Will this sequence of shots really tell our story? Can we be more concise? Where should the cut happen? How's the pacing?
With previsualization, the director can see the big picture and think clearly about the overall cinematic design. By focusing on the staging and cinematics before the character animation, he's able to isolate a dimension of production that really needs attention on it's own.
Previsualization requires an expense of time and manpower that will take resources away from other areas of your project, so the question will inevitably arise, "Do we really need to do this?" The answer is "yes", and the reason is this:
Animation is time consuming and expensive. On RTX Red Rock, we found that, on a good day, we could expect an animator to produce one second of animation per hour. That's eight seconds per day, or 40 seconds per animator per week. (At Pixar, by the way, the weekly average per animator is about 12 seconds.) The previsualization techniques I will outline are fast and cheap by comparison. In the amount of time it takes for an animator to animate one shot, a layout artist will work through a whole sequence of shots. It makes sense to work from the fastest and cheapest, to the slowest and most expensive means of representing the story. Working out all the cinematic issues before animation starts avoids a lot of wasted effort later on. Waiting for a shot to be animated before realizing that it should really be cut, wastes a lot more time than realizing it should be cut during the previsualization phase of production.
Lucas Arts' RTX Red Rock
Three Previsualization Techniques
I will focus on three previsualization techniques that go beyond storyboards for planning your animation. Two of them, the "Animatic" and "Layout", are both part of the Pixar production pipeline. The other is the "Videomatic", which is a technique borrowed from ILM.
1. The Animatic
Storyboards and the Animatic are the first step in making the script visible. They're also the fastest and cheapest way of representing your story visually. Storyboards represent the fundamental actions of each scene: what each character is doing and feeling, and how they are relating to what's around them.
The main limitation of the storyboard is that it doesn't give you a very good idea of pacing. How long will the scene really run? In order to find out, it helps to turn the storyboards into an animatic.
The animatic is created by scanning each storyboard panel into the computer, importing it into a digital editing system and cutting the sequence together in sync with a scratch dialog track. Music and sound effects are often added as well.
The value of the animatic is to give you a better idea of how well the shots cut together, how long your scene is going to run, and if it is going to be entertaining at that pace.
2. The Breakdown Meeting
With the animatic put together, it's time for a breakdown meeting. A breakdown meeting is a meeting of all the individuals and departments that will have responsibility for producing the cutscene. At the meeting, everyone watches the animatic and then has an opportunity to ask questions and raise concerns. By the end of the meeting, everyone should have a clear idea of what will be expected of them, and will have bought off on the plan. This meeting is a critical part of the production process because it is at this point that the wheels of production are really set in motion. Giving everyone involved a chance to express their concerns and affect the way in which the shots are designed (within reason) can avoid a lot of extra work, not to mention resentment, down the road.
Normally the breakdown meeting happens after the animatic is complete. But, it is also possible to have a breakdown meeting centered around a videomatic.
3. The Videomatic
It's not realistic to think that you can work out all the details of staging in the storyboards, or even in the Animatic. Storyboards tend to emphasize the characters attitudes and actions rather than the precise composition of each shot. The gap between the 2D world of storyboards and the 3D world of animation leaves too much room for miscalculation when you're trying to understand how a shot is actually going to look. Layout and the Videomatic, because they happen in 3D worlds, allow you to more precisely visualize how a shot is really going to work. Camera angle, camera movement, shot size, lens choice---all of these decisions strongly affect the dramatic impact of the scene. With both Layout and the Videomatic, you have the opportunity to test these choices before investing valuable animation time.
Layout and the Videomatic directly address the two fundamental questions that face any film director: where will the camera be placed, and how will the characters move in front of the camera?
A videomatic is a live action film shot with a video camera. Videomatics are quick, and a lot of fun to make. To create one, it isn't necessary to get real actors or create special environments; you can shoot it with your team members as actors and your offices as sets. The Bounty Hunter videomatic was staged in the LucasArts office building with members of the production team. The Director of Animation acted as the film director, making the crucial decisions about camera placement and character blocking. After all the footage was recorded, shots were trimmed and cut together on a digital editing system. The result was a narrative sequence that had each shot and shot length timed out. The Bounty Team was able to get a clear idea of how long each sequence needed to be, what shots were going to work and in what sequence they should unfold.
Whether you approach the layout stage with a videomatic in hand or not, the layout process cannot be skipped. In addition to being a previsualization technique, layout is also the first step in the actual production of a shot. In order to get started in layout, you need a script, storyboards, character models, voice, and a 3D set.
In layout, character blocking is done in the simplest way to tell the story. The objective is to set the fundamental marks in the scene that the animator needs to hit, and to get the character movement timed out as closely as possible. While the look of the character blocking is primitive, the marks that are hit for things like head turns, starting and stopping walks, arm or hand gestures etc. are timed very precisely. The idea is that when the shot finally moves from Layout to Animation, the basic parameters will have been set for the animator to work within.
Whereas the character blocking in layout looks crude, the camera work is taken to a high level of precision. Camera angle, lens, shot width (close up, medium shot, wide shot), camera movement etc., are carefully considered. After Layout, when the shot goes to Animation, the camera is off limits to the animator. It's the animator's job to try to conform to the camera work and shot lengths that are set in Layout. If for some reason he can't, then camera changes must be requested by the animator, and the shot sent back to Layout. The idea is to try and nail down the cinematic structure of a whole sequence in Layout and not have the timing and rhythm of that structure broken later on.
As each shot is built, the connections between the shots are worked out. As I mentioned, it's often the case that, once the layout artist begins to put together the shots that seemed to work so well in the storyboards, the reality of the 3D world doesn't match the imaginative world of the storyboard. At this point, adjustments have to be made. Sometime shots are cut. Sometimes shots are added. Sometimes one shot is married to another shot by means of a camera move. Sometimes the staging of the whole scene needs to be reworked. It is rare when the layout artist is simply able to match how things look in the storyboards.
With each build of a sequence in Layout, the director takes a look and gives his feedback. The scene is then reworked and revisions are made until all the shots flow together easily and the timing of the scene is perfected.
At that point each shot file is sent on to Animation. It is important to feel confident in that your layout works, because once a sequence goes to Animation, the cost of revisions goes way up.
Step 3: Optimize
Previsualization allows you the opportunity to see ahead enough to know what you're up against in terms of production effort. With that knowledge, the first question that should be asked is, "Do we have the resources to do this? Have we got the hardware, software, the talent and the time to accomplish what we have envisioned?"
If the answer is no, then you might want to consider contracting a studio that specializes in doing cutscenes to create yours.
If the answer is yes, the next step is to make sure that the efforts of your production team end up making an impact onscreen. Be smart about how you use the resources you have. Don't waste time working on things that don't matter. The third step in producing better cutscenes is to make the most of your production team's efforts by finding ways to optimize your shots.
In-Game vs. Prerendered Cutscenes
There are two kinds of cutscenes, in-game and prerendered. Most of the optimization strategies that I'll talk about apply to prerendered cutscenes. But I'd like to take a moment to talk about the advantages and disadvantages of both.
In-game cutscenes take place in real time in the game engine. The actions of the characters and camera are scripted so that the player watches them like a movie, but the footage is not prerecorded. With in-game cutscenes, set design will be determined by the dictates of gameplay and the level of detail will be limited by the power of the game engine.
One advantage of an in-game cutscene, from a game-efficiency point of view, is that it can start up instantly, avoiding the load time required when playing a prerendered cutscene. Another is that you can use the low poly, in-game characters and sets, as well as in-game lighting and effects , saving production time. Even the low-poly look can be an advantage at times. For cutscenes that pop up in the middle of gameplay, there is something to be said for maintaining a consistent look rather than jumping to higher resolution images. in-game cutscenes are generally easier to make because of the reuse of in-game resources, and the fact that there is no post-production processing or effects rendering required.
One disadvantage of in-game cutscenes is that the level designers will have to be done constructing their levels before layout can start. This can put a serious crimp in your production schedule. Another disadvantage of in-game cutscenes is that they require complex scripting tools and a sizable programming effort to get them working in the game. And then you have the inevitable technical problems and bug fixes. Don't underestimate the time it will take to get in-game cutscenes working in your game.
Prerendered cutscenes are little movies, and they exist as individual frames that are created outside of the game engine. If your cutscene is prerendered and the set is being designed and built specifically for the cutscene, then it makes sense to create a mockup set first. By creating mockups of the sets and staging the scenes there first, you can use the layout process to tell you what parts of the set will be on camera in each shot. With this information you can tell your modelers to build only what's necessary. And by knowing what elements will be in the foreground and which in the background, you can also tell them to what degree to lavish attention on each area of the set. Layout can save a lot of time and money by showing exactly which parts of the environment need to be built and to what degree.
One advantage of prerendered cutscenes is that they don't require any programmer time. Making them is a known process and one which affords the artist a lot more control over the product. Another advantage of prerendered cutscenes is that they look better. The limitations of in-game models, lighting and effects are still such that they impose a real restriction on image quality. Yes, game engines are making great strides, but they're not there yet. With prerendered cutscenes, the only things that keep you from creating film quality animation are your budget, and the skill of your artists.
Model and Texture Optimization
Previsualization allows you the opportunity to optimize your models. One of the first things that one needs to do when beginning production is to go through the script and make a list of all the models that you'll need to tell the story. But a model list doesn't tell you which objects need to be built with a lot of detail, and which can be built more simply. Previsualization gives you that information. Use previs to gauge how much of your model is in the frame and what parts are seen more often than others. Build only what is necessary for the camera. Background objects need not be rezed up to the level of a foreground object. Background objects can often rely on textures instead of heavy modeling, saving render time down the road. Don't overdo it on the resolution of the textures -- use only what will be necessary for the scene. Finally, if the object is off in the distance, it's a good bet that you don't need a 1024x1024 size texture back there.
Keeping the poly count down, and minimizing the size of the textures that you use, saves time not only for those doing the modeling and texturing, but can be a huge time saver when it comes to rendering.
As far back as the storyboarding stage, but especially in Layout, it's a good idea to try to compose your shots so that you create the least amount of work possible for your animators. As I've mentioned already, animation is the most time consuming and expensive part of the process, so any way you can find to save animator time is good.
- Foot Contact. Avoid showing the feet if the character is moving. Foot contact when the character moves is one of the things that animators struggle to make look natural, so you can give them a break by not including the feet in the shot. Of course, by the time the animators start working on the cutscenes, they may already have walk and run cycles for each of the characters, so showing the feet while the character is walking may not be a problem. But showing any kind of special case foot movement should be avoided.
- Singles instead of Multiples. Another way that you can save the animators some time is by avoiding shots with multiple characters. If one character is talking to another one, just show the one that's talking. Even if a character is only listening and reacting, it can take a lot of time to make their reactions look right.
- Watch out for Extreme Close-ups. Going in for a close up on a single character keeps the animator from having to animate more than one character, but be careful about going in too close. Extreme close-ups, while they may be cinematically right for a scene, can reveal inadequacies in your character model. At a certain degree of closeness, low poly models break down, skin and hair textures look ugly, and lip sync weaknesses can become disturbing. Going in for a tight close-up can open up a can of worms that might be best left unopened.
- Choose Camera Angles Carefully. One of the calculations that should go into your staging and camera placement decisions should be how to not show stuff that is going to be difficult to animate.
- Use Sound FX and Reaction Shots. If there is an action that will be particularly difficult to animate (like a car crash or someone slipping and falling) you might want it to happen off screen. Use sound FX and a reaction shot to tell the audience what's going on.
- Shoeleather, and How to Get Rid of it. "Art is life with all the boring parts taken out." The goal of the editor is to find ways to tell the story completely, but without anything extra. The term "shoeleather" refers to unessential coverage of characters getting from point A to point B. In a more general sense, it has come to mean any unnecessary coverage. Obviously, getting rid of unnecessary shots is a great way to streamline your production process. But understanding which shots are essential for telling your story, and which can be dispensed with can be tricky.
Here's an example: You have a shot of a police car screeching to a stop in front of a building. Cut to an interior of an office where the door bursts open and the policemen come storming in to make the arrest. In the cut between the two shots, the police had to get out of their car, make their way from the curb to the office building, go in the front door, run through the lobby and take the elevator up 12 floors before running down a hallway to the office where they break the door down and burst in. But we don't care about that part of the action and the story is perfectly clear without it. That's shoeleather, so we can cut it. By getting rid of those shots, we compress the time it takes for the police to get from the street to the office door. An audience will accept this kind of compression of time if it moves the story along .
In another scene, however, all the steps it takes to get from point A to point B may in fact be critical to the meaning and enjoyment of the story being told. In a suspense thriller, for example, the heroine walks down the dark hallway, getting closer and closer and closer to the door at the end. Will the killer be there waiting? Will he jump out from behind that bookcase? Could he be hidden in a secret panel in the wall? Every moment of her journey is valuable to the story because it gives the audience time to contemplate every gruesome possibility. In this case, if screen time isn't devoted to building tension, the audience will feel disappointed.
It's the job of the director and the editor to decide what's important and what's not. Sometimes it makes sense to compress time as much as possible by eliminating all but the essential story points. At other times, usually at the most critical dramatic moments, it's desirable to stretch time by showing the action as fully as possible.
Step 4: Learn More about Cinematic Design
Previsualization gives you the opportunity to make informed decisions about cinematic structure. That's a good thing, but it raises a bigger issue. In many, if not most, game companies, there really isn't anybody there who has the expertise to make these decisions. Game companies are set up to make games, not animated films. Many of the artistic resources for making a film are there: concept designers, modelers, texture artists, animators, effects artists. But there is one area of expertise that is missing, and that is in the area of cinematic design. The people in the animation studio with the knowledge of cinematic design are the layout artists and film editors. (Animators generally don't know much about cinematics. Animators are actors, not cinematographers.) Game companies don't tend to hire editors or layout artist (or cinematographers from the film industry) because their talents don't really lend themselves to making computer games. But when it comes to making cutscenes, they provide an area of knowledge that, in many cases, is sorely missing. The more ambitious the cutscene, the more important it becomes to have people onboard with the knowledge to be able to shape the overall cinematic design.
Ignorance of the rules of cinematic design is why cutscenes often look weak and amateurish.
All the problems with cutscenes that I listed at the beginning of my talk, are all problems whose solution lies in better cinematic design. Bad cuts, bad camera work, mistakes in screen direction, poor continuity; these are all problems of conception that come out of a poor understanding of film language.
The forth step in making better cutscenes is increasing your knowledge of cinematic storytelling. This topic is much too big