SIGGRAPH: LucasArts, ILM Talk Convergence Lessons

Author: by Brad Kane, San Diego

Film-game convergence has become an increasingly hot topic lately, and at a Thursday morning SIGGRAPH panel, technical leads from LucasArts and ILM discussed how they have now made convergence a reality. The panelists – ILM’s Steve Sullivan and David Bullock, and LEC’s Nick Pavis and Nick Porcino – discussed the process of integrating their successful feature film and video games divisions, and outlined the challenges and successes of the first large-scale studio convergence. While Sullivan has previously discussed this process with Gamasutra, and there was additional background at the recent Hollywood & Games Conference, this is a notable milestone in that key technical staff from both sides of the integration have discussed current progress in public. Collaboration between the film and game divisions has always been a goal of the two companies, the speakers said, and both sides knew they would benefit from the eventual cross-pollination of ideas, sharing of the talent pool, and integrating of the toolsets and pipelines. It’s only been in the last two to three years, however, that collaboration has become a reality, in large part due to the company’s move to the Letterman Digital Arts Facility (pictured above in LucasArts's Xmas card!) in San Francisco. Employees from both divisions now share office pods, cafeteria tables, and common space, and the time has ripened for true integration of the two studios. Moreover, console technology is finally “getting interesting,” and the potential for both divisions to learn from each other began to fructify over the last few years. Tools are now developed jointly between LEC (short for LucasArts) and ILM, and the visual effects folks are learning as much from the game developers as vice versa. (Case in point: more and more run-time technology from LucasArts is being used at ILM, especially in the realms of real-time physics, motion capture, and rendering.) Sharing of Tools, Sharing of Minds The collaborative approach has become embedded in how the two companies organize their efforts. Engineers and artists have been loaned back and forth on recent projects, often staffed into the same office space. A fixed production staff oversees the shifting teams, allowing for a managed approach to cross-pollination. However, this is about more than sharing assets and people. The two companies are unifying their technologies and techniques in a big way. Tools are developed in collaboration, with the goal of sharing them across multiple pipelines, without maintain multiple versions of any single tool. Techniques, scripts, and workflow concepts are equally shared, leading to the new-routine presentations and user groups within the company. The panel demonstrated a number of tools and advancements that have sprung from the collaborative process. Among those demonstrated: Zeno: This standard 3D content creation tool is at the heart of the pipelines for both ILM and LEC, allowing artists to customize their workflows and leverage show-specific scripts and plug-ins. User-developed plug-ins and widgets can be shared between divisions. Zed Level Editor: The Zed Level Editor is used as both ILM’s scene management system and as a level editor for games. The tool was designed based on experience from both the film and games divisions of the company. Effects Authoring: Combining ILM’s particles architecture and data flow editor with LEC’s optimization knowledge and GPU tricks has allowed for sophisticated effects production. Animation Logic and Blending: Again, taking ILM’s expertise in human motion and character animation technology, in combination with LEC’s run-time blending, GPU acceleration, and understanding of game character states, has led to live, deep control over all kinds of animations. Similar innovations have been made in Terrain Editing, Likeness Capture, Lighting, Shader Network Editing, Facial Animation, and cinematic scene design. Third Generation Game Development LucasArts again discussed its forward strides via strategic technology partnerships, aimed at allowing the division to accomplish new creative and technical goals - as demonstrated at multiple industry events recently. Simulation-based gameplay is high on the studio’s agenda, and they’re using NaturalMotion’s “Euphoria,” a real-time animation/AI engine that also uses Havok Physics underlying it, for several upcoming releases. Simulation-based play was demonstrated via a procedurally-animated Indiana Jones rolling down platforms, crashing through crates, and dropping from the sky, all created via Euphoria. Another area of development involves Pixelux, a tool that simulates the molecular structure of materials, allowing for the procedural manipulation and destruction of any object according to its material type. In the demo, a crystal and a dinosaur skeleton were procedurally destroyed, collapsing and breaking just as one would expect them to do. The demo also showed a forest of organic alien plants, procedurally animated with Pixelux. Other areas of development include real-time cloth simulation – the demo showed a character running around in a black cape, simulated as one complete piece that collides with both itself and the character – innovations in which are now being brought back over to ILM as a pre-visualization tool – and a new real-time rendering engine, re-designed to meet artistic goals for the upcoming “The Force Unleashed” (rather than allowing the engine to determine what was artistically possible.) A Brief Convergence Postmortem All told, the speakers agreed, integrating the film and games divisions has been a challenging but mutually rewarding process. They attributed their success to a number of key factors: o The team got an early start and had time to learn from their mistakes. o Ramp-up efforts were focused on a single game. o They built a successful vendor/client workflow for tool iteration and general communication. o Tracking and project management practices have matured at ILM, and are influencing the structure of game teams. o The group established a solid build/release/QA process, which is now counter-influencing ILM. o The unified teams continue to collaborate without factionalism. o LEC directly leveraged existing ILM workflows, tech, and training, which is now filtering back into ILM. At the same time, the integration process has not been free from obstacles. A few of the areas that needed fixing along the way: o Underestimating the cultural challenge involved in integrating teams from both visual effects and video games. o Focusing on a single game confused structural issues, i.e. who drove what, what’s shared across the studio versus worked out per-game, etc. o Lines of decision weren’t always clear, and multiple stakeholders meant that issues took longer to decide than they needed to. o Overhead of developing small tools was too high, especially in contrast to the general toolsets typically used by the game development team. o Weak coordination between engine programmers and tools programmers. o LEC was too exposed to raw ILM code checks, and vice versa. Lastly, the panel addressed next steps and areas that can be addressed now that a functional base toolset is in place: o Increase engineer speed and the efficiency of iterating on tools, particular gameplay-related tools. o Look for more explicit “win-win” solutions between games and film, and exploit them when possible. o Now that the toolsets have converged, push for more artist collaboration. o Make more of LEC’s runtime technology available to ILM and Lucas Animation. o As common tools mature, begin to reduce the number of per-game tools engineers. o Leverage LucasFilm’s previz system as a tool for creating cinematics, and possibly use it as a production tool. o Continue adaptation of other tools, e.g. Zeno. o Continue evolving structures for local, shared, and core technology in a multi-game, multi-project, multi-division company. Conclusion Overall, it was an interesting and rather honest panel, shedding light on the growing potential of convergent development between games and film. LucasArts and ILM are in a unique position given their single roof, shared licenses, and combined decades of experience in the two industries, but the continuing of video game hardware growth and general technical innovation will continue to push these two industries together. It will be interesting to watch as other companies follow the lead of these two innovators, and to see how and when an industry-wide move toward film/game convergence begins to really take shape. [UPDATE: 08/10/07, 12.35pm: Clarified that Natural Motion's character animation/AI technology also uses underlying physics from Havok.]