Author: by Adam Schnitzer

Cutscenes, in one form or another, have been a part of computer games from almost the beginning. There were the crude, pixelated animations of early arcade games, the live-action, bluescreen productions of PC games, and now the 3D animation of today's consoles. Purists like to say that cutscenes are a distraction, an unnecessary and annoying intrusion on the integrity of the game. Gameplay is king! And, even those who enjoy cutscenes would have to admit that there are times when they get too much of a good thing. Some games have so many cutscenes, or such long cutscenes, that you have to wonder whether the game designers had a movie in mind instead of a game. And then there are those games with such boring, confusing, ill-conceived cutscenes that it would have been better if they hadn't bothered making any cutscenes at all.

When you leave the interactive world of the game and enter the cinematic world of the cutscene, the rules change. You depart the continuous, spontaneous world of on-the-fly camerawork, and enter the world of traditional cinematic structure. As someone who has worked in both the game industry (at LucasArts) and in the film industry, (at Pixar) I look at cutscenes in games with a keen awareness of cinematic design. Shot compositions, staging, cutting, pacing: every aspect of what is seen in the cutscene has expressive importance and is there, exactly the way it is, because someone decided that they should be.

The rules by which these decisions are made are the rules of the language of film. Unfortunately, game cutscenes are often conceived in ignorance of the basic grammar of this language. And when you have a shaky grasp on the rules of grammar, it's hard to express yourself clearly.

There are some very beautiful cutscenes out there, and there are lots of bad ones. What are the problems that I see?

There are lots of ways that cutscenes can go wrong. What follows are four fundamental steps that can help in creating cutscenes that go right.

Step 1: Know Your Purpose

The first step in making better cutscenes is being clear on what those purposes are.

Step 2: Previsualize it

The second step in producing better cutscenes is to do better planning. In this case, another name for "planning" is "previsualization". To understand how previsualization fits into the production pipeline, it's worth taking a moment to look at how the whole animation production process works.

The Pixar Production Pipeline

Cutscenes are made the same way that any animation is made. It doesn't matter whether the animation is being produced for a game of a cartoon, there is a certain order that things generally need to go. As an example of a basic model of animation production, I'd like to outline the way things worked at Pixar when I worked there on A Bug's Life, Toy Story 2, and Monsters Inc.

Notice that there are a lot of things that happen ahead of the animation step. To use a theatrical analogy, animation is the part of the process when the actors step onto the stage and perform. But before we have the performance, we need the rehearsals. In the animation pipeline, "previsualization" is the "rehearsal".

Previsualization is where you can answer all the critical filmmaking questions. How should the action be staged? Where should the camera be positioned? How should the shot be composed? Will this sequence of shots really tell our story? Can we be more concise? Where should the cut happen? How's the pacing?

With previsualization, the director can see the big picture and think clearly about the overall cinematic design. By focusing on the staging and cinematics before the character animation, he's able to isolate a dimension of production that really needs attention on it's own.

Previsualization requires an expense of time and manpower that will take resources away from other areas of your project, so the question will inevitably arise, "Do we really need to do this?" The answer is "yes", and the reason is this:

Animation is time consuming and expensive. On RTX Red Rock, we found that, on a good day, we could expect an animator to produce one second of animation per hour. That's eight seconds per day, or 40 seconds per animator per week. (At Pixar, by the way, the weekly average per animator is about 12 seconds.) The previsualization techniques I will outline are fast and cheap by comparison. In the amount of time it takes for an animator to animate one shot, a layout artist will work through a whole sequence of shots. It makes sense to work from the fastest and cheapest, to the slowest and most expensive means of representing the story. Working out all the cinematic issues before animation starts avoids a lot of wasted effort later on. Waiting for a shot to be animated before realizing that it should really be cut, wastes a lot more time than realizing it should be cut during the previsualization phase of production.

Three Previsualization Techniques

I will focus on three previsualization techniques that go beyond storyboards for planning your animation. Two of them, the "Animatic" and "Layout", are both part of the Pixar production pipeline. The other is the "Videomatic", which is a technique borrowed from ILM.

1. The Animatic

Storyboards and the Animatic are the first step in making the script visible. They're also the fastest and cheapest way of representing your story visually. Storyboards represent the fundamental actions of each scene: what each character is doing and feeling, and how they are relating to what's around them.

The main limitation of the storyboard is that it doesn't give you a very good idea of pacing. How long will the scene really run? In order to find out, it helps to turn the storyboards into an animatic.

The animatic is created by scanning each storyboard panel into the computer, importing it into a digital editing system and cutting the sequence together in sync with a scratch dialog track. Music and sound effects are often added as well.

The value of the animatic is to give you a better idea of how well the shots cut together, how long your scene is going to run, and if it is going to be entertaining at that pace.

2. The Breakdown Meeting

With the animatic put together, it's time for a breakdown meeting. A breakdown meeting is a meeting of all the individuals and departments that will have responsibility for producing the cutscene. At the meeting, everyone watches the animatic and then has an opportunity to ask questions and raise concerns. By the end of the meeting, everyone should have a clear idea of what will be expected of them, and will have bought off on the plan. This meeting is a critical part of the production process because it is at this point that the wheels of production are really set in motion. Giving everyone involved a chance to express their concerns and affect the way in which the shots are designed (within reason) can avoid a lot of extra work, not to mention resentment, down the road.

Normally the breakdown meeting happens after the animatic is complete. But, it is also possible to have a breakdown meeting centered around a videomatic.

3. The Videomatic

It's not realistic to think that you can work out all the details of staging in the storyboards, or even in the Animatic. Storyboards tend to emphasize the characters attitudes and actions rather than the precise composition of each shot. The gap between the 2D world of storyboards and the 3D world of animation leaves too much room for miscalculation when you're trying to understand how a shot is actually going to look. Layout and the Videomatic, because they happen in 3D worlds, allow you to more precisely visualize how a shot is really going to work. Camera angle, camera movement, shot size, lens choice---all of these decisions strongly affect the dramatic impact of the scene. With both Layout and the Videomatic, you have the opportunity to test these choices before investing valuable animation time.

Layout and the Videomatic directly address the two fundamental questions that face any film director: where will the camera be placed, and how will the characters move in front of the camera?

A videomatic is a live action film shot with a video camera. Videomatics are quick, and a lot of fun to make. To create one, it isn't necessary to get real actors or create special environments; you can shoot it with your team members as actors and your offices as sets. The Bounty Hunter videomatic was staged in the LucasArts office building with members of the production team. The Director of Animation acted as the film director, making the crucial decisions about camera placement and character blocking. After all the footage was recorded, shots were trimmed and cut together on a digital editing system. The result was a narrative sequence that had each shot and shot length timed out. The Bounty Team was able to get a clear idea of how long each sequence needed to be, what shots were going to work and in what sequence they should unfold.

Layout

Whether you approach the layout stage with a videomatic in hand or not, the layout process cannot be skipped. In addition to being a previsualization technique, layout is also the first step in the actual production of a shot. In order to get started in layout, you need a script, storyboards, character models, voice, and a 3D set.

In layout, character blocking is done in the simplest way to tell the story. The objective is to set the fundamental marks in the scene that the animator needs to hit, and to get the character movement timed out as closely as possible. While the look of the character blocking is primitive, the marks that are hit for things like head turns, starting and stopping walks, arm or hand gestures etc. are timed very precisely. The idea is that when the shot finally moves from Layout to Animation, the basic parameters will have been set for the animator to work within.

Whereas the character blocking in layout looks crude, the camera work is taken to a high level of precision. Camera angle, lens, shot width (close up, medium shot, wide shot), camera movement etc., are carefully considered. After Layout, when the shot goes to Animation, the camera is off limits to the animator. It's the animator's job to try to conform to the camera work and shot lengths that are set in Layout. If for some reason he can't, then camera changes must be requested by the animator, and the shot sent back to Layout. The idea is to try and nail down the cinematic structure of a whole sequence in Layout and not have the timing and rhythm of that structure broken later on.

As each shot is built, the connections between the shots are worked out. As I mentioned, it's often the case that, once the layout artist begins to put together the shots that seemed to work so well in the storyboards, the reality of the 3D world doesn't match the imaginative world of the storyboard. At this point, adjustments have to be made. Sometime shots are cut. Sometimes shots are added. Sometimes one shot is married to another shot by means of a camera move. Sometimes the staging of the whole scene needs to be reworked. It is rare when the layout artist is simply able to match how things look in the storyboards.

With each build of a sequence in Layout, the director takes a look and gives his feedback. The scene is then reworked and revisions are made until all the shots flow together easily and the timing of the scene is perfected.

At that point each shot file is sent on to Animation. It is important to feel confident in that your layout works, because once a sequence goes to Animation, the cost of revisions goes way up.

Step 3: Optimize

Previsualization allows you the opportunity to see ahead enough to know what you're up against in terms of production effort. With that knowledge, the first question that should be asked is, "Do we have the resources to do this? Have we got the hardware, software, the talent and the time to accomplish what we have envisioned?"

If the answer is no, then you might want to consider contracting a studio that specializes in doing cutscenes to create yours.

If the answer is yes, the next step is to make sure that the efforts of your production team end up making an impact onscreen. Be smart about how you use the resources you have. Don't waste time working on things that don't matter. The third step in producing better cutscenes is to make the most of your production team's efforts by finding ways to optimize your shots.

In-Game vs. Prerendered Cutscenes

There are two kinds of cutscenes, in-game and prerendered. Most of the optimization strategies that I'll talk about apply to prerendered cutscenes. But I'd like to take a moment to talk about the advantages and disadvantages of both.

In-game cutscenes take place in real time in the game engine. The actions of the characters and camera are scripted so that the player watches them like a movie, but the footage is not prerecorded. With in-game cutscenes, set design will be determined by the dictates of gameplay and the level of detail will be limited by the power of the game engine.

One advantage of an in-game cutscene, from a game-efficiency point of view, is that it can start up instantly, avoiding the load time required when playing a prerendered cutscene. Another is that you can use the low poly, in-game characters and sets, as well as in-game lighting and effects , saving production time. Even the low-poly look can be an advantage at times. For cutscenes that pop up in the middle of gameplay, there is something to be said for maintaining a consistent look rather than jumping to higher resolution images. in-game cutscenes are generally easier to make because of the reuse of in-game resources, and the fact that there is no post-production processing or effects rendering required.

One disadvantage of in-game cutscenes is that the level designers will have to be done constructing their levels before layout can start. This can put a serious crimp in your production schedule. Another disadvantage of in-game cutscenes is that they require complex scripting tools and a sizable programming effort to get them working in the game. And then you have the inevitable technical problems and bug fixes. Don't underestimate the time it will take to get in-game cutscenes working in your game.

Prerendered cutscenes are little movies, and they exist as individual frames that are created outside of the game engine. If your cutscene is prerendered and the set is being designed and built specifically for the cutscene, then it makes sense to create a mockup set first. By creating mockups of the sets and staging the scenes there first, you can use the layout process to tell you what parts of the set will be on camera in each shot. With this information you can tell your modelers to build only what's necessary. And by knowing what elements will be in the foreground and which in the background, you can also tell them to what degree to lavish attention on each area of the set. Layout can save a lot of time and money by showing exactly which parts of the environment need to be built and to what degree.

One advantage of prerendered cutscenes is that they don't require any programmer time. Making them is a known process and one which affords the artist a lot more control over the product. Another advantage of prerendered cutscenes is that they look better. The limitations of in-game models, lighting and effects are still such that they impose a real restriction on image quality. Yes, game engines are making great strides, but they're not there yet. With prerendered cutscenes, the only things that keep you from creating film quality animation are your budget, and the skill of your artists.

Model and Texture Optimization

Previsualization allows you the opportunity to optimize your models. One of the first things that one needs to do when beginning production is to go through the script and make a list of all the models that you'll need to tell the story. But a model list doesn't tell you which objects need to be built with a lot of detail, and which can be built more simply. Previsualization gives you that information. Use previs to gauge how much of your model is in the frame and what parts are seen more often than others. Build only what is necessary for the camera. Background objects need not be rezed up to the level of a foreground object. Background objects can often rely on textures instead of heavy modeling, saving render time down the road. Don't overdo it on the resolution of the textures -- use only what will be necessary for the scene. Finally, if the object is off in the distance, it's a good bet that you don't need a 1024x1024 size texture back there.

Keeping the poly count down, and minimizing the size of the textures that you use, saves time not only for those doing the modeling and texturing, but can be a huge time saver when it comes to rendering.

Animation Optimization

As far back as the storyboarding stage, but especially in Layout, it's a good idea to try to compose your shots so that you create the least amount of work possible for your animators. As I've mentioned already, animation is the most time consuming and expensive part of the process, so any way you can find to save animator time is good.

Step 4: Learn More about Cinematic Design

Previsualization gives you the opportunity to make informed decisions about cinematic structure. That's a good thing, but it raises a bigger issue. In many, if not most, game companies, there really isn't anybody there who has the expertise to make these decisions. Game companies are set up to make games, not animated films. Many of the artistic resources for making a film are there: concept designers, modelers, texture artists, animators, effects artists. But there is one area of expertise that is missing, and that is in the area of cinematic design. The people in the animation studio with the knowledge of cinematic design are the layout artists and film editors. (Animators generally don't know much about cinematics. Animators are actors, not cinematographers.) Game companies don't tend to hire editors or layout artist (or cinematographers from the film industry) because their talents don't really lend themselves to making computer games. But when it comes to making cutscenes, they provide an area of knowledge that, in many cases, is sorely missing. The more ambitious the cutscene, the more important it becomes to have people onboard with the knowledge to be able to shape the overall cinematic design.

Ignorance of the rules of cinematic design is why cutscenes often look weak and amateurish.
All the problems with cutscenes that I listed at the beginning of my talk, are all problems whose solution lies in better cinematic design. Bad cuts, bad camera work, mistakes in screen direction, poor continuity; these are all problems of conception that come out of a poor understanding of film language.

The forth step in making better cutscenes is increasing your knowledge of cinematic storytelling. This topic is much too big for me to try to address in the time remaining. The bibliography at the end of this article lists some books on the subject.

In the meantime, I'd like to focus in on one topic that that touches on the subject: how certain traditional shooting and editing techniques can be used to solve the problem of how to enter and exit a cutscene.

Out of Gameplay, and Back Again: Making the Transition

Moving the game player from gameplay to a cutscene, and back again, is always a little tricky. One obvious problem is that you are moving from an experience in which the player motivates the camera, to one in which the game designer is in control. Another is that you are often moving from a camera style of continuous action, unbroken by a cut, to a more traditional, cinematic style with cuts.

Filmmakers have developed strategies that help the audience move from one shot to the next, across the cut, with minimal confusion. The place where there is maximum potential for an audience to become confused is on the cut. Establishing a stage line, being consistent about screen direction, cutting on the action, maintaining an eyefix ---these are all ways of getting from shot to shot smoothly and efficiently. The goal is to be able to make cuts that are unnoticed by the audience. The best kind of a cut is the kind you don't see.

One of the central questions of cinematic design is where and when to make a cut. And, in computer games, the cut that occurs between the gameplay and the cutscene is the trickiest cut of all, because of the potential for discontinuity. The reason that there is such a risk of jarring discontinuity is because it is often hard, as a game designer, to know where a character is going to be when the cutscene is triggered. Gameplay being what it is, we are not always in complete control of the exact pose of a character at the moment when the transition will occur.

If handled badly, this discontinuity across the cut will cause confusion in the player. The player may see a jarring pop as the character jumps from his ingame position to his cutscene position. In film when a character "jumps" from on position to another across the cut, this is called a "jump cut".

Of course, this kind of problem comes up again and again with cutscenes. Fortunately, there are strategies that we can borrow from film that have evolved to deal with exactly this issue.

Each of these strategies, cut-ins, cutaways, lead-ins and hand-offs, allows you to introduce the main subject of the shot indirectly, and in the process hide any discontinuity between where the character is ingame and where he is in the cutscene. Whether you are moving from gameplay to cutscene, or from cutscene to gameplay, the same principle applies.

A Note on Camera Technique

In the world of 3D animation, the camera is free from the constraints of the real world. This is both a blessing and a curse. Nothing gives CG animation that cheesy 3D animation feel more that a camera that flies around all over the place at lightning speed.

One of the reasons that we feel this way is because we are all very familiar with the conventions of the film language. Having spent thousands of hours in our lives watching films, we've absorbed a sense of how cameras should move. Pixar's philosophy is to make the audience feel at home by mimicking the way that real world cameras move and operate.

For example, traditional motion picture cameras are big and heavy. In order to move them around, production companies lay down track and hire grips to push the camera along the track on wheeled dollies. When the camera starts it's motion, it takes a moment for the dolly grips to get the camera to come up to speed. Then, at the end of a move, it takes a moment to slow to a stop. To mimic this, we put plenty of ease-in and ease-out on the translation of the 3D camera in order to give it the proper feeling of weight. Dolly shots, crane shots, pans, tilts, hand helds, and so forth, are all modeled on the way these shots feel in traditional live action films.

Having said all this, I would add that it would be foolish not to take full advantage of the freedom that one has with a CG camera. A camera that takes up zero space and is completely unaffected by gravity can be a distinct advantage. But be careful, because camera technique that is uninformed by a knowledge of traditional cinematic practices can often lead to a feeling of disorientation in the audience.

Follow The Steps

In order to build a better cutscene, the first step is to be clear about the purpose of doing cutscenes at all. Step two is to do plenty of planning, by means of thorough previsualization, as a means of rehearsing the cinematic structure of your animation and making sure it tells your story clearly and dramatically. Previsualization also allows you to do Step 3, which is to optimize each shot and use your artistic resources more effectively. And, finally, Step 4 is to acquire a greater knowledge of film and cinematic design, not only for creating compelling and professional looking cutscenes, but also for navigating the problems of moving from the world of gameplay to the world of visual storytelling.

Bibliography

______________________________________________________