Google Docs for Game Engines, and more.

Google Docs is a great tool for real-time collaborative document editing. Most of us have used it at some point. Automated versioning and online cloud storage are just some of the features included, and it is the go-to tool for writers, publishers and teams that want to work together on a document.

Where Google Docs is a great tool for real-time editing, tools such as Shotgun and Ftrack are very useful for reviewing assets. Shotgun and Ftrack are screening room tools, where filmmakers can review, annotate, compare, and select scenes to be part of a movie. Shotgun has slowly made its way to game development as its reviewing and annotating feature helps to point out potential problems within an art asset.

Now imagine the features of Google Docs: real-time editing and automated versioning, combined with the features of a Screening Room tool: reviewing, selecting, comparing, and combining asset versions within a Game Engine Environment and associated Game Development Tools. Doesn’t this sound fantastic? Is it even possible? If it is possible, then how would it work? What would the features of such a tool look like? Hopefully I will be able to explain all of these questions in this article.

The Juicy part (just one of many scenarios)

Let’s say, persons A, B and C want to work together on a level. Option one, they can grab three chairs and sit at one desk. Option two, start working on their own and try not to intervene in each other’s work, hoping that later merging will work. Or option three they can start real-time collaboration. Seeing each other’s changes in real-time, without the danger of messing up each other’s work or the work that others did. All work they do is automatically saved to a local dedicated server or stored securely in the cloud, so no work is ever lost.

All actions done within the scene are stored safely in a database, making it possible for members to do unlimited undos. Besides this, if another person wants to join in and collaborate, they can simply connect to one of the users already in collaboration mode and help out.

Now, at a later stage the art director may walk by and say: “I like the sky that we had before and I do not completely agree with the forest you have built there; the trees do not fit with the scenery”. “No problem!”, you say, while you click the tree the art director was mentioning. The tool will pop up all combinations that have been made of that tree by the different artists in the team. You move through the different combinations and select one, then click on the sky and change it back, also with the tool, to the version the art director had advised. This can all be done within a few seconds whilst the art director is still at your desk.

You discuss some other changes that may benefit the level. Heck, maybe even go back to where you started collaboration and try out a different route. With two versions of the same scene it is now possible to compare them through the tool. Maybe the end product is a combination of the two scenes where you selected parts to be combined.

After some discussing, you send the end product to the producer, who will also be able to change some elements in the level through the use of the tool. The final product is then sent to the client for final approval.

All in a day’s work.

This is only one scenario that might arise when using a tool like this. There is even the possibility to send a part of the project to clients and let them play around with the different asset versions. It could even be that a specific asset in the game breaks, and instead of reverting the entire project to a previous version, just replace that particular asset.

Current Engine Tech

Game engines use all kinds of different techniques to make collaboration possible. Some use build in solutions others use 3th party software to handle it. Take for instance Perforce which is a versioning engine, that is introduced in a lot of different studios. This tool makes it possible to lock a file. When a file is locked only the game developer that locked it can use the commit action on the file. This insures that for instance a scene does not get corrupted because multiple developers were working in them. Not really a collaborative features, nonetheless, it allows big teams to work on the same product without messing up each other's work.

Currently Engines, such as Unreal or Unity, solve scene merge problems by layering the scene into different sections. Unity, before version 5, did not support this, and scene merging was almost impossible without the help of 3rd party tools, such as Cahoots or Perforce. Unity is trying to solve this with Unity Collaborate which is currently in Beta.

Unity Collaborate (image 0.1) is a tool that allows developers to share their project within the cloud. Because of the cloud functionality, new team members from all around the world can be added to a project, and no team member will ever be out of sync. It also solves the scene problem by working on an asset basis, meaning that individual asset manipulations will be stored, not the entire scene. A conflict will occur only if one object is manipulated from two different sources at the same time.

Image 0.1: Unity Collaborate

Unreal solve the collaboration problem by allowing artists and designers to merge different scenes together into one scene. This makes it possible for versioning engines to override that scene without compromising the work of others in different scenes. It still uses versioning engines such as Perforce to synchronise the scene across clients. However, it is still a in engine solution for solving the problem.

Tech / System Architecture

Implementing Google Docs features within a game engine requires a lot of back-end processing to make sure everything is running smoothly. Even imagining the number of files saved to the server and the data overhead gives me goosebumps. First thoughts were to use an existing versioning engine, such as Perforce or Git. However, as these were already established tools and would require a large amount of customisation, building it ourselves with the use of certain third party technologies looked to be the best choice. In this way, we could even learn a thing or two about versioning engines and the problems they were facing in the development of these tools.

The first version is currently built in Unity with a Virtual Machine running on the operating system CentOS 7 for the back-end. I am using a Mongo database for all data storage. Locally I store JSON data for when users are unable to connect to the server (image 0.2). Unity is connected to a DLL file that handles all interaction with the server. The reason for this is that the DLL can then be connected to other game engines or tools that are being used for game development. All functionally that is not engine specific will be programmed in this DLL file. Think of writing and reading JSON data, adding listeners for server messages, login and logout handlers, etc.

Image 0.2: System Architecture

Challenges and/or Limitations

Merging

The automatic branching and versioning engine can give troubles when merging files. In traditional versioning engines, we already have a large number of asset versions. The production workflows in these systems usually follow a straight path, with some small branches here and there. The server in this case will always have the latest version of the product. Perforce even has a name for this called the Single Source of Truth (Knop S.E, 2016. Link: https://www.perforce.com/blog/161207/perforce-helix-game-developers-%E2%80%93-distributed-single-source-truth).

However, if you introduce an automated versioning engine on an asset level, the amount of branching can become quite complex. We are not going to merge individual files together, there are software packages that are far more capable of doing this. However, merging user's action and creating a timeline for this action could be very useful, think of feedback sections or slowly moving through different changes.

A timeline could be generated from two files or from collaboration paths. This can then be combined to form a new scene, model or whatever the case would be. The problem arises when one action is removed from that chain. The question is then whether the timeline will still perform the same output? Could the large numbers of files capture this distortion and create a still valid outcome? That is something that we need to test, check and research a bit more.

Dependencies

Game projects are extremely complex in the sense that assets and components are connected to other assets and components. Removing one could break the whole system, it is dangerous and could become a disaster when done by an automated system or an ignorant user.

As a solution, we could combine dependent assets together; reverting one will revert the others. This will make sure that connected assets will always be moved together. The system could also give the suggestion for moving them together, and it is then up to the user for making the decision.

What does the Future hold?

In a couple of weeks, we will start our continuous build process. We will put together a test environment in which participants can use the tool and where data can be collected. From that point we can analyze and change the tool according to the discoveries and see if the tool is beneficial and what limitations it has. We can then modify the tool and do another testing round to see if improvements were made and if we solved the issues that occurred in the last test.

Summary

Google Docs and Screening Room tools are used in a large variety of circumstances. Moving them from their own environment into a Game Engine environment is quite a challenge, but not impossible.

Automated versioning makes sure that files are never lost, the system even stores the changes between file versions. The latter makes it possible to rebuild files by the action that took place, walking through every step and every change. Making it possible to review the build up of a file, instead of only the end result. A use case for this would be in game development educations, which now really on the end result of the students. With this tool feedback can be given on specific action the student has performed. There are some issues with developer’s privacy which need to be taken care of, however, the benefits way out the negatives.

The abundance in file versions makes it possible to select from a large number of different possibilities. Making game development move from a latest version process to an asset combination version process.   

The drop in, drop out features allows developers to connect to each other’s game engine and to start working together to try out that thing they wanted to do. Don’t worry about files being used by other developers; you can work simultaneously together because a branch is created automatically. Check which parts of which version work best and disregard the others.

Many Thanks

You may think I am mad or violently disagree with everything I stated here, or maybe I overlooked a crucial part in Game or Versioning Engines that makes the tool obsolete. Let me know! I would love to hear about it.

It would be rude to not thank you for taking the time to read this article, I really appreciate it. I am always eager to hear people's thoughts, and if you have any questions, remarks or just want to say hi, do not hesitate to mail me ([email protected]).