Making an entire game in Tilt Brush

This was also posted on the Out Of Bounds Games blog.

Since the start of the current VR revolution I have been more interested in how the technology could be used for creating games rather than just playing them. It should come as no surprise then, that one of my favorite VR apps are Tilt Brush, the 3D painting program that allows for a completely new way of drawing that gives depth to your doodles.

I finally got my own Vive a couple of weeks ago. Since Ludum Dare was around the corner, I decided to make a game where all the art was drawn in Tilt Brush. I thought it would give the game a unique look, and also be a fast way to create 3D content, an important aspect when doing a game jam. You can find the game called The Trans-Interdimensional Laboursaving Transportation Service on Ludum Dare’s website, as well as on Game Jolt and Itch.io. I couldn’t find anyone else who had tried to use Tilt Brush this way, so I decided to write this post on how I did it and my takeaways from the process.

I feel like I’m being watched...

The setup

Turns out the pipeline to get Tilt Brush sketches into Unity, which is the engine I use, was very straight forward. From inside Tilt Brush you can export an FBX of your sketch. I thought I would have to write my own shaders, but it turned out the Tilt Brush devs already provide a set of basic shaders as well! You still have to manually assign this shader to the imported models in Unity so I ended up writing a small AssetPostProcessor script that would automatically handle this when the FBX file was copied into the project folder which made it easier to adjust the look of the game since all brush strokes of the same type would share the same material. This would sometimes still break when Unity had to split up the imported mesh because of it’s 65k vertex limit per mesh but it still saved some time. Here is the source if you want to check it out yourself.

Terrain, some flowers, and sleepy blobs painted in Tilt Brush and rendered in Unity

What worked well

The game ended up with the unique look I was after. You can clearly see it’s been drawn by a person which I think gives it it’s own charm. So aesthetically I’m very pleased with the result. Since you can’t really animate a Tilt Brush sketch other than its rotation, scale and location I choose to make a game, where you see a lot of different static environments which worked well.

If you want people to play your jam game it’s a very good idea to make a build that can be played on the web. I was really surprised how compact the WebGL build was. I haven’t gotten any complaints about the game relying on beefy GPUs to run either which was a nice surprise as well.

Lastly my assumption that it would be a quick method for making art turned out to be mostly correct. Drawing the art inside Tilt Brush was quick, but there was a lot of hurdles getting it into Unity as explained in the next section.

What didn’t work so well

The shaders provided by the Tilt Brush devs only cover the most basic brushes. All the more VFXish of them such as fire, lighting and disco are not supported.  You don’t get normal maps either, so some brushes such as Ink and Oil look flat when imported into Unity. There’s also no shader to to get the non lit strokes of the Marker brush to show up correctly. I know It’s a simple shader to write, but at the end of the jam I was too sleep deprived to sort it out ,so I ended up just using the premade shaders and calling it a day.

As you can see the shape of tapered strokes also didn’t match how they look inside Tilt Brush. It’s controlled in the shaders with alpha cutoff, but there wasn’t a value that made them have exactly the same shape as inside Tilt Brush.

There’s some positive news in the horizon though! The team behind Tilt Brush are going to release a set of resources that should make it a lot easier to import your sketches into Unity in the near future so really looking forward to give that a go.

The other main hurdle is that Tilt Brush is really more geared for sketching than for actual content creation. It’s designed to be approachable and playful to interact with. As a result it doesn’t have many of the features you would expect from a program appealing more to professional users. For example you can’t group your sketch into layers making it tricky to delete strokes. The biggest issue though was not being able to set a custom pivot point for exported drawings as shown on the picture below.

Tell me of the pivots of your homeworld Usul

I compensated for this the standard way with using a GameObject parent as the pivot in Unity, but it was something that I had to spend more time on than I would have liked. Other

features I could also have used, was to just export a part of a sketch and overwrite existing sketches from inside Tilt Brush. Tilt Brush has never crashed on me but I didn’t want to take any chances so I saved fairly often which filled up the gallery really quickly.

Where Tilt Brush also excels is really in creating organic shapes with free hand brush strokes. My concept relied on creating a bunch of objects with harder edges which proved to be fairly time consuming. There is an option to use a straight edge tool, but it took a lot of trial and error to get the stroke orientation correct.

I spent way too much time on drawing the hard edges of the box the player is placed inside while playing the game.

Conclusion

I really enjoyed making the art for my game with Tilt Brush. Even though getting the sketches into Unity isn’t a perfect process yet it was already a very powerful and quick way to create 3D content. With the new tool chain promised by the Tilt Brush devs on the horizon it will only get better. Next time I’m doing a 3D entry for a game jam I’ll likely use this approach again.

PS!

I’m planning to organize an online game jam for making games with Tilt Brush around February next year after the new tool chain has shipped. If you think that would be awesome to participate in follow me on Twitter or Facebook to stay tuned!