[In this technical article, Cinemaware and EA veteran Khawaja -- the creator of the Fork Particle visual effects middleware -- looks at the state of UI and flow for game tools, suggesting practical tips to make your own internal tools and scripts easier to use and succeed with.]

Today, the creative process in various aspects of game development requires trying out numerous ideas and multiple iterations to perfect the final product. A fast iteration process enables better results.

Real-time visual feedback or real time preview is the key to fast iterations during video content development. This principle lends to development of software tools with user friendly graphical user interfaces (UI) and an added level of sophistication.

A digital artist once said to me, "Half my life was spent watching the progress bar on my computer monitor." We have come a long way from that point in time. Now, we need to create a much larger amount of assets for our games, so efficiency in creating these assets is more critical than ever.

Where the Problem Lies

Growth in video game production content translates into development of improved tools to efficiently create and manage content assets. These tools -- both externally licensed ones and those created internally by teams to manage particularly custom tasks -- incorporate data creation for new technologies and streamline production pipelines.

Eventually, tools may intelligently design some assets automatically. Currently, the creative process requires multiple iterations to reach the final asset revision.

There are two developer issues associated with the basic problem. The first issue emerges because often not enough time is spent on tools during a game project's pre-production period. It is also frequently assumed that enhancement to the software tools during the project will be sufficient.

This generally results in patched tool features and inflexible software architecture that eventually can only be fixed or improved by performing an overhaul. But due to time restrictions, an overhaul cannot usually be done for the project the tool was originally written for.

As for issue two, an interactive software tool usually requires a powerful engine under the hood, and friendly UI. It is natural to implement the underlying engine first because it is the tool's core functionality. It can be sophisticated and complex, which takes time to put together.

However, the lack of sufficient time assigned to the user interface design and implementation can compromise the use of the tool and limit the speed of work. It can also limit the number of users because of the steep learning curve, and that can be especially painful during project crunch periods.

In the final phase, the asset content is integrated so it can be reviewed in the context of the game for look and feel. A slow asset integration process results in fewer revisions -- or none -- due to its cumbersome nature, which can obviously compromise final quality.

A Better Way

Time spent in the project to improve software tool user interface and reduce integration effort can result in a highly productive and creative environment -- not to mention the frustrations it can save the design team.

An intuitive graphical user interface is generally standards-friendly, requires minimal steps, and exposes all options at a given stage in the workflow to complete creation of an asset. The user interface for tools with real time visual feedback adds a level of implementation complexity; however, the benefits to the user far outweigh the implementation effort.

The chart in Fig. 1 shows a step-by-step method for creating an efficient graphical tool. The first step toward UI design involves investigating current software applications and then conducting user interaction and look-and-feel studies. It is an observational study of commonly used software applications and interactive software tools.

We don't want to re-invent the wheel with UI design. Commonly used controls and interaction sequences will reduce the application usage learning curve.

Fig. 1

A user interaction study allows you to get a sense of user interactive behavior and lists UI controls that feel natural to the user. Such a study also highlights the importance of observing the look and feel of commonly-used software tools. That is what the user is already familiar with, so it will increase the comfort level and reduce the learning curve.

Furthermore, consistent UI design is crucial. If the user learns to use a particular set of UI controls scheme then a consistent design will automatically make it easy to use many features of the entire interface.

A user interface look-and-feel study lets you gather ideas for your application framework layout. Primarily, you need to observe what is standard or familiar to end users. Secondly, the layout should allow easy navigation to the tool's feature UI categories. In addition, workflow has to be kept in mind, so the user can sequentially go from one stage to the next during the tool use process.

Consistency in user interface look may be more important than the quality of the style itself. However, a customized look such as button icons and cursors help to achieve and retain user interface familiarity or associate a user's interaction learning with that interface. It also brings a level of user confidence in the professionalism behind the making of the tool.

Real-Time UI Design

UI design standards have matured over time and sometimes define what is intuitive. Design for real time UI is directly associated with the progression of the standards. But on the other hand, these standards must be molded to fit the needs for real time visual feedback.

The underlying principle to real time UI design is to have instant update or instant preview enabled on actual viewed, active content at all times and display the confirmation button only when really needed. The tool will update attributes as soon as the user changes them. Secondly, the design should be intended to reduce as many number of UI button clicks as possible, at least in the most frequently-used sections. There is also no real need for duplicated methods and UI to do the same operation.

Although UI for real time updates doesn't always follow the standards, it mostly only reduces the user interaction steps and does not generally introduce replacement or new steps. In other words, the user will still be seeing standard interface components such as buttons, lists, and text boxes. A user may follow standard steps to discover that some steps are no longer required. Here are a few examples of how user interface controls can be used for faster update.

Fig. 2

Data Entry. There are many instances where values can be updated using a slider instead of an edit box, Fig. 2. You may need to provide control over range if the range cannot be fixed. The nice thing about sliders is that values can be updated as the slider moves along, so the user will be able to see changes at each tick.

Color Picker. The standard color picker can be modified to update on mouse roll over or mouse button down. This way color update can occur while the user moves the cursor over the color swatch. Mouse roll over in the color swatch window will display the color on the content being modified or edited. The user will then click once to confirm the selection.

Edit Box. Not all numeric value entries can be converted to sliders. However, change events can be used to make updates effective as the user types in the new value. This way the return key press or update button press is not needed as an additional step.

Pull-Down List Box. An update can occur as a selection highlights in the dropdown list. The user will easily preview content with the different selections and be able to quickly pick the desired one.

Drag & Drop. This is another method that makes property assignment very simple. An example is to drag a property object and drop it on the asset being edited. The property object would instantly associate itself with the content and show the update instantly.

Software Architecture Considerations

Tool software architecture includes foundation features, framework, extendibility, and data format. Fundamental features should be listed during the technical design stage -- that can be critical to the software tool's architecture.

These may include Undo, Redo, Cut, Copy, and Paste, which require serious technical design considerations because they may apply to most parts of the software. Fig. 3 diagram shows data flow between components of the application.

In the framework category, these are some of the key application components. The user interface drives the application and may be kept as much decoupled from other components as possible. This may not be as simple as it sounds, but it will have advantages in maintenance down the line.

The current UI state such as dialog sizes, positions, and control states can be stored in an XML definition. This definition can be stored on disk so that the application can maintain its UI state when launched the next time.

The Undo and Redo state stores the state of content data, difference in content data, or command history. Again, this state can be stored on disk or in memory so that Redo and Undo actions can be performed.

The Content Data module manages the attributes data to include load, store, and conversion. The Live Update module is responsible for sending the data over to the remote target or game application when a change in the data occurs.

Fig. 3

As for extendibility, it is all too often that new features need to be added to the software, and it may not be done by the original authors. In some cases an open architecture or external module plug-in based architecture can be highly beneficial.

Ideally the attribute data format would not require a conversion to reflect object display. In other words, the data directly maps to content attributes. This may not be possible for all attributes, but data that is directly mapped will make real time update of the attributes easier.

For example, the color of light in the scene modified from the user interface will directly update the data field in the asset attributes data state without a conversion process.

Implementation

Content attributes data has to be mapped directly to user interface controls and data update has to reflect the change immediately. Ideally, the real time engine will be using the data and therefore, change in the data will be reflected automatically. Care must be taken when modifying data that has resources associated with it such as textures and geometry data.

The new resource may need to be newly created and the old resource disposed of. There will have to be checks on use of resources that are being disposed of in the tool source code to avoid unwanted results while new resource is prepared. A safe method would be to halt any display and not use the disposed resource until new resources are ready.

Let's go through a simple example to show implementation of user interface controls mapped directly to an object attribute. As shown in Fig. 4, we have a slider that controls the spin speed of an object being displayed in the viewport.  I use pseudo code (I'm keeping it similar to C++) to demonstrate data structure and code implementation.

Fig. 4

The slider mapping information is kept in a searchable database or a simple array, which has the structure shown in Fig. 4a. The Slider ID is a slider control resource identifier and pAttribute is a pointer that points to the address of the object attribute.

Fig. 4a

In the Fig. 4b example, the object attribute Spin Speed is kept in a separate data structure. This data structure is used directly for object display. Here the SpinSpeed field is the angular rotation speed of the object in the viewport. Since this field is being used in every frame cycle to retrieve the rotation speed of the object any changes to this field will result in instantaneous update in rotation of the displayed object.

Fig. 4b

As shown in Fig. 4c, when an object is selected or becomes editable, the searchable slider mapping information database is initialized or updated to reflect the current mapping. The pAttribute field is assigned to the address of object's spin speed field. This way the attribute value can be updated by using a pointer to it.

Fig. 4c

When the slider control changes its update, event updates the value of the attribute it is mapped to, Fig. 4d. The slider value is assigned to the object attribute by using the mapped pointer. The object data structure is decoupled from the slider database. Yet, the attribute update is reflected in the object attributes and onto the object display in real time.

Fig. 4d

Resources like device objects such as textures and geometry and other properties' data that requires additional processing cannot be directly mapped to the UI control.  Those controls may only update reference information such as file name or unique identifier which is then used to prepare the resource associated with the object.

For example, if a new material is assigned to the object then the previously assigned material is freed and the new material is created and prepared to be assigned to the object. Once assigned, the material is reflected in the viewport.

Live Update

Live update is more than just icing on the cake because it speeds up the final step in the iteration process. It helps asset integration and lets the user perform final tweaks to the assets in the game application at run time. After integration, the final tweaks can be an interactive and iterative process itself and having live update speeds that up.

One solution for Live Update is using the local network. If data is reasonable size, then it is sent directly to the game application memory.

Data mapped directly to the content asset attributes will reflect the update automatically. However, an update trigger can allow the game application to perform any conversions or resource updates to safely update the attributes.

If data is large, then it may be stored on disk and the live update trigger will inform the game application to load the modified data from disk. This is relatively slower, but it will do the job faster than having the user manually trigger an update on the game side.

Fig 5 shows how identical data structure is sent to the game application for update.

Fig. 5

An update performed on the tool side should update the target application. This will require a live connection. Or if the connection is fast then a connection can be opened when data is transferred and then closed. This way there won't be an order to launching the tool and target application.

Conclusion

Tools are one of the most important components for production efficiency. Good tools allow faster iterations, which helps achieve better quality in game design and content. A tool's user interface plays a big part in friendly and efficient usability.

Ultimately, real time visual feedback gets the fastest turnaround time in game content production and provides leaps in freedom to create quality assets.

Implementation of such tools requires a methodological approach that may take longer than usual. However, its architecture and user interface design principles will be invaluable to the production teams for current and future projects.

---

Title photo by fdecomite, used under Creative Commons license.