Unreal Game Sync - Migrating to Internal UE4 Builds

I decided to write up my recent experience migrating from a launcher installed version of Unreal Engine 4 to an internal build because the documentation is extremely sparse and it took me a lot of time to put together the information needed to perform what turned out to be an extremely simple task. 

What is Unreal Game Sync and why should I use it? 

Unreal Game Sync is a tool built by epic to interact with perforce to aid in developing Unreal Engine 4 projects. It has quite a lot of features on top of p4v but the primary use, to my mind, is distributing internal engine and project builds.

If you started developing UE4 in a launcher build and want to migrate to a build you built yourself, you'll need to tackle the issue of distributing those builds without requiring anyone who works on the project to build the engine themselves, the simple way is to dump the entire engine into your version control system and that works well if code changes are rare and broken builds are a minor inconvenience but If you can't risk breaking the editor with your changes you will need to use something like UGS.

Unreal Game Sync solves this by letting you build the engine once, zip it up and store the zipped binaries in perforce, where UGS when it pulls a new changelist will also fetch the corresponding engine (& game) binaries and automatically extract them into place (in theory) ensuring the binaries are always in sync with the content.

Migrating to an internal build of UE4

UGS expects your project to be set up a very specific way and epic (or anyone else as far as i can tell) didn't document what that is. If you started developing with a launcher version of the engine, your project will be in a folder on it's own with a .uproject file and various folders for binaries and content. The uproject file is a text file and inside the text file there are various settings, one of those settings is the EngineAssociation setting, which is used to determine which copy of the engine to load the file with. If it's a launcher build it will read something like '4.23' and if it's a locally built build it will be a generated guid.

The problem with this is that every machine will have a different guid, the best solution for internal builds is to remove the EngineAssociation field. When there is no EngineAssociation, opening the uproject file will first search up it's directory tree to find an engine and if one isn't found will prompt the user with an engine selection dialogue.

So we want to remove the EngineAssociation and place the engine files downloaded from github above the project file in the perforce repo. i.e. The engine will be placed at -

and the game project will be located at -

On top of this I had to mark the following file as writable in perforce, since the build touches it -

Once we have our perforce repo in this form, we can run the Setup.bat file which will be used to download the dependencies, I opted to not store the dependencies in perforce and instead require it to be run once on each machine, I'm not certain this was the best approach since it doesn't scale effectively to large teams, pushing and maintaining the dependencies in perforce may be a better approach.

After that we need to run GenerateProjectFiles.bat and start up visual studio, if it's working as intended, the UE4.sln will also have the game code as well as the engine code, the old game vs solution file won't be required any longer. This approach allows you to view, step into and debug engine code while debugging the game. From here you can build the engine.

Once the engine and game is built you should be able to start up the locally built editor by double clicking the .uproject file. Before we go on to setup UGS, we should build the zip archive and push it to perforce and ideally set up a continuous integration system to automatically build and push new binaries, I used jenkins to do this but there's plenty of documentation on CI solutions and it's outside the scope of this article. The command for building and pushing the archive can be found on the UGS reference page here -

UGS Reference

Before running this script on a machine you will need to make sure the Windows SDK has the Windows Debugging Tools component installed as the build process uses pdbcopy, you can install the WDT by modifying the Windows SDK install via the Settings -> Apps -> Apps and features panel in windows, locating the Windows SDK and selecting modify.

The script and target options define the process for building, zipping and uploading builds if there have been code changes, the EditorTarget is the name of the editor .target.cs file that is in the vs game project, there are more options too including a GameTarget option for building the standalone game too.

The ArchiveStream is the location you want to place the zipped binaries. It wasn't immediately clear to me how epic wanted me to structure this perforce stream and so I'll go into a bit more detail about how I believe they intend it to be laid out. Epic require the zipped binaries to be placed in a stream depot in perforce, I believe they want the binaries in their own mainline stream (in this case named Dev-Binaries), I'm not certain why they'd require this since it doesn't seem to be the kind of thing you'd branch but who knows ¯\_(ツ)_/¯.

Setting up UGS

UGS is a tool, which epic distribute as source code along with the engine, the vs solution can be found in Engine/Source/Programs/UnrealGameSync

You need a link Epic Games account to see it on github

Building and running UGS is fairly well documented in the UGS Reference.

UGS uses an odd update mechanism where there is an installer that installs a launcher that fetches the tool from a hardcoded perforce path. So before building the code, you need to install wix 3.8 and change the DefaultDepotPath string in DeploymentSettings.cs to point to a perforce location where the UGS launcher can find updated binaries for UGS. Once done build the installer and UGS. Commit UGS to perforce and then install the launcher, if it's working correctly, you'll be able to startup UGS and point it at your .uproject file and it will show you your changelists.

From here you should be able to turn on the 'Sync Precompiled Binaries' option and any changelists which have a corresponding binary zip in perforce will not be greyed out.

Before rolling the tool out everyone who doesn't have visual studio will still need to run Setup.bat and install the latest vcredist because the one installed by Setup.bat will probably not be the one you built the engine against and will be out of date.

If you're trying to minimise distruption the last step will be pushing the modified .uproject file to source control along with any updated plugins, allowing for a seamless transition from the installed build to the internal build.

Conclusion

Hopefully this information can help shorten the UGS setup time for others as the whole process turned out to be fairly simple but I had to work out quite a lot of it myself due to the absence of documentation and hopefully if anyone has better ideas for how to integrate or streamline all this stuff please let me know in the comments or on twitter.