With the increasing number of assets and people involved in game projects, manually maintaining game assets takes on an ever-increasing portion of the project. In order to reduce, and hopefully eliminate, this time from future game projects, Ensemble Studios decided to evaluate its own asset management needs and implement a system for storing and managing all game assets. In this article, Herb Marselas discusses how they translated their asset management needs into an effective asset management system for future games, and the technologies utilized in doing so.
Age of Empires II: The Age of Kings consisted of more than 40,000 game and production assets, ranging from bitmaps and textures to 3D models, sounds and music, and source code files. However, with the exception of the source code, managing game assets at Ensemble Studios has largely consisted of editing, copying, and renaming files on local and shared network drives. This process has sometimes resulted in a number of problems, including misplacement, corruption, or accidental loss of game assets. All of these problems result in effort that must be spent finding or re-creating missing assets.
With the increasing number of assets and people involved in game projects, manually maintaining game assets takes on an ever-increasing portion of the project. In order to reduce, and hopefully eliminate, this time from future game projects, Ensemble Studios decided to evaluate its own asset management needs and implement a system for storing and managing all game assets. The purpose of this article is to discuss how we translated our asset management needs into an effective asset management system for future games, and the technologies that we utilized in doing so.
Needs of the Many
During the development of previous Ensemble games, game assets were managed using a directory structure that was centrally located on a network server and copied to the user's workstation as needed. Assets were edited locally with final changes copied back up to the server or edited directly on the server itself. To indicate successive revisions of an asset, incremental numbers were sometimes added to the end of the filename.
This combination of local and server files created confusion when two users both attempted to work on the same shared server asset, or they made different local versions that were later copied back up. It was also difficult to determine which older revisions of assets were truly good enough to keep and which could be thrown away.
However, even with these potential problems, there were several big advantages to a centrally located, directory-based asset system. The first advantage was that the servers hardly ever went down. There were few times during the course of developing Age of Empires II: The Age of Kings (AoK) when server or network problems disrupted access to game assets.
Another definite advantage was simplicity. Every user was already familiar with copying files between Windows folders. Updating an asset on the server, or adding an asset, or putting a new asset into the game was as simple as copying between Windows folders. We were able to use the pros and cons of the current directory-based asset management system to create a list of requirements for the new asset management system that built on the positives but removed the negatives.
Beyond these requirements, a new asset management system had to be able to handle a file of virtually any size, as art and sound files can range in size up to hundreds of megabytes. Another requirement was that the new system should be based on serving the asset to the user's local workstation for editing. This was especially critical, as our main 3D content package, 3D Studio Max, had problems editing files across the network.
The new system also had to be capable of exporting a complete set of game-ready assets from those under asset management. This would remove the onus from the project teams of trying to verify that they really had copied the very latest version of an asset into the appropriate game directory. As long as the latest revision of the asset was in the asset manager, they could be assured it would get in the game.
The final requirement was for a simple workflow system to help the art team keep better track of where assets were in the art pipeline. The workflow system would have three nodes, allowing an asset to be tracked from prototype, to ready for game use, to finalized.
Having established the requirements for the new system (Figure 1), we then faced the looming question: Build or buy?
A Single Solution?
Because the programming team was already using Microsoft SourceSafe 6, the first task was to examine the viability of using it as an asset management system for the whole team. While SourceSafe offers a good user interface, a stand-alone version, and an API to create tools to interface with it, a number of concerns arose immediately. The biggest of these were issues of dealing with files, and even a moderate number of users.
With only 15 programmers using it, our SourceSafe system was having performance and consistency problems that resulted in a number of hours spent each month in maintenance and recovery. Working with Microsoft support, we found that the problems we were experiencing affected some number of SourceSafe sites with no discernable cause. Other problems with SourceSafe included a severe performance problem when attempting to check in large assets (anything bigger than about 10MB), even when the files were stored directly, and there was confusion when a user had files checked out on more than one workstation. Although we chose not to use SourceSafe, it did have two features that we could not ignore: its simple user interface and seamless integration with Microsoft Visual C++. These became our guidelines for usability in selecting an asset management system (see Tables 1 and 2).
Table 1: NT-based asset management products
COMPANY | PRODUCT | REPOSITORY | API? | ASSET SERVER? | LIMIT ON TOTAL BYTES OF ASSETS? |
|---|---|---|---|---|---|
Bulldog | Two Six | Relational Database (RDBMS) | Yes | Yes | Limited only by RBDMS |
eMotion | Cinebase3 | RDBMS | Yes | Yes | Limited only by RBDMS |
Filemaker | Filemaker Pro | RDBMS | Yes | Yes | Limited only by RBDMS |
Microsoft | Visual SourceSafe 6 | File-based | Yes | No | Yes, 4GB |
Merant | PVCS | Proprietary database | Yes | No | Not documented |
NxN | Ailenbrain | RDBMS | Yes | Yes | Limited only by RBDMS |
In our quest to try to keep the whole team using a single asset management tool, we also evaluated Merant PVCS. With PVCS we came to some of the same conclusions that we had reached with SourceSafe. While PVCS is a good tool for simple programming projects, we found it to have many of the same shortcomings as SourceSafe for game asset management. It also lacked a good, intuitive stand-alone UI.
Finally, we looked at NxN's Alienbrain. Alienbrain is more like an asset management toolbox than an off-the-shelf asset manager. This meant that if we did use it, we couldn't just drop it in and go. We would have to learn how to use its interfaces, then build an asset management system on top of them.
Table 2: Is there a single solution?
PRODUCT |
|---|
SourceSafe |
PVCS |
Three other asset management products we looked at briefly were eMotion's Cinebase 3, Bulldog Two.Six, and Filemaker Pro. In general, these are all competent asset management tools. However, they suffer from the same issues that plague the other products we reviewed in greater depth -- a lack of front-end integration and workflow. They are potentially more powerful back-end solutions depending on your need, but in the end a lot of time will be spent creating a custom solution relying on their individual APIs.
The Final Solution
The biggest issues for all these tools, however, are the lack of front-end integration and even the simplest type of workflow. If we were to use any of these asset managers, we would have to learn their APIs and then spend the time creating a workflow system and integration with front-end tools such as 3D Studio Max.
Because we already had expertise in the design and architecture of large-scale databases, we decided to spec out how long the implementation would require if we created an asset management system from scratch. We estimated the time needed to create our system would be approximately six man-weeks of programming time. This did not include time for testing or for major new requirements or features that cropped up during development, testing, and deployment.
The asset management system we ultimately implemented consisted of four major components. The largest component was the relational database on the back-end to track and manage the assets, and the client-side data access layer to it. On the client side, there were two user interface components. One was a plug-in (Figure 3) to integrate the asset management functionality seamlessly with Max, and the other was a stand-alone front end for those users. The final component was an exporter used to create game builds from the latest versions of assets.
Back-End and Client
On the back-end, we chose to use Microsoft SQL Server 7 as the central database (Figure 2). Oracle or Sybase would have worked just as well, and may be required if we increase the number of users significantly. However, SQL Server 7 running under Microsoft Windows NT 4.0 is enough computing power to handle our current and future needs at this time.
We also decided to store the asset revisions directly inside SQL Server as binary large objects (BLOBs). We could have stored them on a network drive, but we felt that SQL Server could stand the additional load, and storing them in the database provided additional security.
To improve performance of the database, especially with the large number of assets stored inside it, the database server was configured with two ultrawide SCSI controllers. Each controller then supports two ultrawide SCSI hard drives. This configuration allowed us to place the database system files and logs, the small asset management data, the large BLOBs, and the index data on separate drives. This improves performance by allowing the database to spread its access patterns across all four drives. Database security itself is handled directly through the Windows NT domain user authentication system. This means that granting users database access is as simple as adding them to one of the existing NT domain groups.
On the client side, we built a data access layer using ODBC. This gave us several advantages. First, it's simple to maintain. It's also easy to learn. Moreover, there is no dependency on bound data controls (such as MFC), which allows us to actually build an in-game connection to the back-end SQL Server by just adding the additional link to the ODBC libraries.
All of the asset manager's client functionality was then created in a single layer on top of ODBC and other core technologies (Figure 3). The specifics of the Max plug-in and the stand-alone user interface are then abstracted into separate files on top of the core functionality. This creates a system where all of the code is completely shared through most of the two separate user interfaces.
Regardless of whether the user accesses the asset manager through Max or the stand-alone user interface, both systems present the asset manager to the user with just a Windows Explorer interface (Figure 4). Because all of the users are already familiar with navigating a tree structure, this has significantly reduced the users' learning curve when using the system.
Keep the Artist in Max
During the development of AoK, the art process consisted of a number of individual steps. Once an artist had created a unit for in-game use, it then had to be rendered out, postprocessed, then added to the game by a lead artist or designer. This meant that it could be a long time between the time artists worked on an asset and when they actually saw it in the game.
As we started implementing the asset management system, our mantra became "keep the artist in Max." In other words, give the artists everything they need to create, manage, and view their models and textures in the game directly in Max. The only time they should have to leave Max is to run Adobe Photoshop, which they can also launch from a button in Max.
This mantra led to the creation of two additional Max plug-ins to support our asset management system. The first was a texture/bitmap browser that allowed the artist to search, view, check out, and use any bitmap asset stored in the asset manager (Figure 5). The texture browser was built as a Max extended viewport plug-in using MFC and Lead Tools imaging control. The Lead Tools provided us a very powerful yet simple-to-use ActiveX component that could be used directly in the MFC dialog box that formed the basis of the texture browser.
The second plug-in was also a Max extended viewport -- it was actually the entire game running inside Max as a viewport. This plug-in was relatively simple to create by making a version of the game that built as a .DLL rather than an .EXE file. Some additional code was required in the window handling code to compensate for the fact that the game was now running as a child window and had to interact with the Max input system.
A small amount of functionality was also required to allow the artist to specify which of the objects in a Max scene should be put into the game. Instead of creating a cumbersome communication system between Max and the game, the artist's scene is exported to a file in a format that the game can load. The game is then told to load the file and display its contents. Because this is the whole game engine running inside Max, the artist can examine the model in the context of other existing game assets and scenarios. As complicated as getting the game engine itself to run within Max was, creating both extended viewports was very straightforward compared to managing Max files and seamlessly integrating with Max itself.
Max files themselves might initially be located in any local or server directory, and they may refer to texture files that exist in any of the Max texture paths. Adding a Max file to the asset manager meant moving the file to its managed directory, then scanning the file for any texture files and moving those to the same directory as well, and finally updating the paths of textures referenced by the Max file.
If the Max file contained in-game models, not only was the Max file added to the asset manager, but an in-game version of the file was generated and stored in parallel. This allowed the back-end export process used to create the game build to be much simpler. It just had to copy data out of the database and store it in files.
Every texture also generated one or two additional files on check-in. A thumbnail image was created and stored to support the texture browser, and textures used in-game automatically generated a texture in the in-game format.
Integrating with Max
Integrating directly into the Max user interface and menu system was one of the hardest challenges we faced in creating a seamless asset management system (Figure 6). Unlike the Microsoft Visual C++ IDE, Max does not have a well-defined interface for asset management tools to plug into. Also, it's not possible to create menu items in Max using the Max SDK or MaxScript.
Because we wanted to make the integration as seamless as possible, we had to rely on Win32 programming to manipulate the main Max window and menu system directly. Although this appeared to be a complicated solution, it gave us the flexibility we desired to create new menu entries, and the ability to override and enhance existing Max functionality.
To facilitate this integration, we used a Max general utility plug-in (GUP) to glue Max and the asset management system together. The GUP is one of a number of DLL-based plug-ins that Max supports for modifying or extending existing functionality. Before reading the following explanation of how we integrated the GUP plug-in directly with the Max menu system, you may want to download the source code from the Game Developer web site at http://www.gdmag.com/.
The Max plug-in architecture is based on deriving developer-implemented classes from base Max C++ classes. In this case, our MaxUIModGUP class is derived from the Max GUP class. Max identifies plug-ins in two ways: through their file extension (.GUP for a GUP plug-in), and with a simple class factory called ClassDesc.
When this DLL plug-in is built, we change the file extension from .DLL to .GUP and place it in the MAX PLUGINS\ directory. As Max scans for each set of plug-ins it recognizes, it knows that this plug-in is at least used as a general utility plug-in. Once loaded, Max uses four simple functions to interrogate the DLL. LibDescription returns a simple text description of the plug-in. LibNumberClasses returns the number of class factories (or ClassDescs) in the plug-in. LibVersion is the version of Max that the plug-in will work with. And most importantly, LibClassDesc returns an instantiation of our own derived version of ClassDesc called MaxUIModClassDesc. Max can instantiate our MaxUIModGUP class using the MaxUIModClassDesc::Create member.
In the case of some plug-ins (for example a viewport plug-in), this class factory could be called multiple times. For a GUP it is only called once at startup. This is why we can simplify the code somewhat by using global variables to store our flags and state information.
Once instantiated, the MaxUIModGUP::Start member function is called. MaxUIModGUP has access to the main Max window handle using the inherited member function MaxWnd. Once we get the window handle, we can subclass it with our own message handler (SubclassWndProc) and return success. Subclassing the window ensures that we get the Max window messages before its message handler does.
In implementing our scheme to add new menu entries and enhance existing functions, we only care about two Windows messages: WM_INITMENU and WM_COMMAND. WM_INITMENU is sent to the window when a menu is about to become active. This allows us to look for the main menu, and modify its functionality. However, we must be careful to modify only the main menu, and then to modify it only once.
Because Max creates a number of menus, we use GetMenuItemCount and GetMenuString to make sure the menu we're getting is the main menu. Once we've ascertained that we have the correct menu, the modifyMenu function inserts two new entries into the file menu. For this sample, we're adding a menu option that will force a complete redraw, plus a separator to make the menu look pretty. In the end, DrawMenuBar is called to make sure that the menu is properly updated when it draws.
Back in the SubclassWndProc function, we need only add the WM_COMMAND case to look for the new menu entry we created (IDC_MAXMENUMOD), and process it accordingly. One additional piece of functionality that has been added to this WM_COMMAND handler is a wrapper for the Max File Open menu entry. This code stores off the current Max project filename, calls the default handler to open a new file, and then displays a message box to inform you if a new file has been opened.
Overriding the File Open function in this way may seem a bit dangerous, because the File Open menu entry might not be 0x9c43 in a future version of Max. Unfortunately, there's no other viable way to add a very important piece of asset management functionality: detecting when a user opens a file that he or she doesn't have checked out from the asset manager. While you can register a callback using the Max function RegisterTimeChangeCallback to determine if the current filename has changed, you don't receive the notification until after the file is opened. The Max notification system may look like another alternative, but it's just that: a notification. You can't stop or change something already in progress.
The asset manager needs to have a priori access to File Open requests for several reasons. If the user doesn't have the file checked out, it will be read-only. It might also not be the latest version of the file. Either way, if the file is under asset management the latest version of the file needs to be copied down to the user's workstation and made writable if someone else doesn't already have it checked out.
Beyond Asset Management
In addition to the texture browser and game viewport, we added a number of other features beyond basic asset management. One of the simplest and most useful features was the versioning of Max plug-ins. File information is stored on the server about the current versions of Max plug-ins, including the asset manager itself. If a plug-in is out of date, the user is notified upon starting up Max. This has helped in many situations where the user has accidentally installed an older plug-in when reinstalling a workstation. Additional functionality was added to allow the asset manager to update itself automatically when new versions are posted internally. This has eliminated the need for the user to worry about keeping track of the latest version and its location. All users need to do to get the latest version is to restart Max.
The three-node workflow system helped us understand the state of game assets more clearly (Figure 7). New assets are marked "not ready for game." When an asset is ready to be exported to the game, it becomes "ready for game," and then finally "final." An asset can stay in any node as long as is necessary.
One of the benefits of using SQL Server directly was that we could create stored procedures to enhance asset workflow. One example of this is a configurable notification system that automatically e-mails the game designer using MAPI (Messaging Application Programming Interface) the first time a game asset becomes ready for the game. This takes the burden off the creator of the asset of having to remember to send a separate e-mail, and ensures that the designer is kept aware of when assets become available.
There is also a branching system so that an asset that is either "ready for game" or "completed" can be branched to the next lower node. This allows the artist or designer to leave the last version that they know is good enough for the game to continue to be exported, while allowing them to go back and potentially make major revisions. Once the user is ready for the latest version of the asset to be exported again, the asset is unbranched (Figure 8).
Asset types were added to allow the teams to classify new assets added to the system. Some of these asset types, such as "Mesh (in game)" or "Texture (in game)" are backed by code in the asset manager that performs special functions. In these two cases, in-game versions of the assets are created and checked in whenever the original assets themselves are checked in.
Future Work
While our asset management system has solved a number of problems that we have experienced in managing game assets in the past, several areas still need additional work. The hardest area that we have yet to address is the issue of the "build machine" -- the versioning of the tools themselves. We've already run into this problem once in creating a patch for one of our previous games almost a year after it shipped. The original game was compiled using Microsoft Visual C++ 5. Our current compiler is Visual C++ 6 with Service Pack 3.
Getting the game to finally compile correctly with the latest version of the compiler took several days. While the new compiler certainly improved the code quality and allowed us to find some problems that the old compiler had missed, we don't know what additional problems the new compiler might have introduced.
It seems that the only way to fully version-off the tools that were used to create the game is to set a workstation in the corner that contains the tools and the shipping version of the source code. Any other solution that involves storing the program install disks away and having to reinstall old software probably will not be utilized.
One of the other problems to solve is being able to regenerate all the in-game versions of assets from the original versions stored in the asset manager. Although we try to make sure that we never break existing game assets, there usually comes a time (sometimes more than once, unfortunately) where all of the art, sound, or levels for the game need to be reprocessed. With the original data, and the information on how it was exported for the game stored in the asset manager, it's possible to write a batch system that can perform this reconversion. One of our next goals is to actually implement this using the Max DCOM sample code to create our own batch system to reprocess assets as necessary.
Summary
In the end, it would have been easier if there were an off-the-shelf asset management solution that met our needs right out of the box. Those asset management systems directed towards game and content creation all seem competent in delivering basic asset management functionality, but they are sorely lacking in the areas of front-end and third-party integration and workflow.
For us, developing an asset management system completely in-house was well worth the effort. We ended up with a system that met our needs, and because we fully understand the design and architecture, we can continue to add functionality as needs arise.
The total staffing time was about eight full-time man-weeks of programming, and three part-time weeks of testing. The programming time did run over by a couple of weeks, but this was mainly due to adding new features during development. It should also be noted that the programming time would have taken significantly longer if we hadn't already had back-end server design and implementation experience in-house.
Overall, the biggest problems revolved around our inability to modify and integrate with the Max user interface system more simply. The whole Max user interface needs to be based on MaxScript, and MaxScript itself needs to be much more powerful in order to do what we need to do. An even better solution would be if Max provided a specified asset management interface similar to Microsoft's Visual C++, thereby allowing third parties to create asset management solutions for it.
Acknowledgements
Creating an effective and useful asset management system was a team effort. Thanks to everyone involved throughout the process for remembering those little details. Thanks also to everyone at Ensemble Studios for reviewing this article.
For More Information:
MSDN
Felder, Ken. "Microsoft Visual SourceSafe OLE Automation." Microsoft Developer Network. Microsoft Corp., October 1995.
http://msdn.microsoft.com/library/techart/msdn_vssole.htm
Web Sites
Bulldog Two.Six
www.bulldog.com
Discreet 3D Studio Max
www.discreet.com
eMotion Cinebase3
www.emotion.com
Filemaker Pro
www.filemaker.com
Lead Tools
www.leadtools.com
Microsoft SQL Server
www.microsoft.com/sqlserver
Microsoft Visual SourceSafe 6
http://msdn.microsoft.com
Oracle
www.oracle.com
Sybase
www.sybase.com
Merant PVCS
www.pvcs.com
NxN Alienbrain
www.alienbrain.com
Read more about:
FeaturesYou May Also Like
