Written by Max Röhrbein-Kling and Johannes Kuhlmann

This is part two of our series of blog posts about our experience with bringing Galaxy on Fire 3 - Manticore to Vulkan.

Our posts follow this structure:

When we started working on the Android version of our game, we decided to use Vulkan for rendering (there is also an OpenGL ES version, but that is not of interest here). This series is about our experience with implementing a Vulkan renderer and getting it to work on different devices, in particular on those running Android. So, we are mainly going to talk about the most interesting aspects of our implementation and then dive into what we learned along the way.

As we already did at the beginning of our first post, we would once more like to point out that the focus of our Vulkan renderer was to ship a game. Consequently, its design is rather pragmatic and we cannot promise that the things that worked for us will also work in another context. We do believe, though, that our implementation is still reasonably versatile and well done.

This second post is about the special considerations that went into adding support for textures, shaders and graphics pipelines during the implementation of our Vulkan renderer.

Textures

A texture, or rather VkImage, in Vulkan has the VkImageLayout property, and many operations require images to be in specific layouts. You can change an image's layout with a VkImageMemoryBarrier. There is VK_IMAGE_LAYOUT_GENERALwhich supports most operations, but

Therefore, you are better off just not using VK_IMAGE_LAYOUT_GENERAL at all.

A fairly simple example of how you can manage image layouts is the way we deal with textures. Here the idea is that there is a setup phase during which the texture is initialized. At the end, the image is transitioned to VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL for sampling. The texture stays in that layout until the texture is destroyed.

The setup phase is straightforward as well. The image is in VK_IMAGE_LAYOUT_UNDEFINED when its memory is bound. From there, it is transitioned to VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL which is a requirement for the vkCmdCopy()command(s) for getting data into the image to work. In the end, we do the before-mentioned transition to VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL and afterwards the texture is ready to be used.

For images used in render targets, this approach will not quite work though. In our case render targets alternate between two layouts: writing (VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL) and reading (VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL). To stick as close to our approach with textures as possible, we decided to keep the render targets in the read layout by default. Only when a render pass binds that render target for rendering into we transition the render target into the write layout. We then transition the render target back to the read layout immediately after the pass writing to it is done.

Shaders

When planning for Vulkan support, keep in mind that you will also have to touch your asset pipeline. The main reason for this is that Vulkan does not accept shaders in the form of raw source code. According to the specification, a Vulkan implementation only has to support SPIR-V.

Before you can load a shader into Vulkan, it has to be precompiled by some kind of SPIR-V precompiler. A big advantage here is that your shaders are at least syntax-checked during the asset build and not when they are - at some point - loaded into the game. Another advantage is that it reduces loading times for shaders.

We use Google's Shaderc to precompile our shaders as it is rather easy to integrate and works well. The input here is GLSL which allows us to share our shader sources between OpenGL (ES) and Vulkan.

Another thing Vulkan does not support is reflection for shaders. With other graphics APIs, the approach often is to load a shader and then query it to find out what its inputs (and possibly outputs) are. With Vulkan, you now have to find another approach.

If you have simpler shaders or if your shaders are just a few in number, you may be able to get away with just knowing, and hardcoding, that your shaders take specific inputs.

If you have lots of shaders or maybe shaders you do not know about in advance, you will have to find another way to get the reflection data. We currently use SPIRV-Cross for this purpose. This library makes it easy to examine the precompiled SPIR-V code and retrieve the required information. This is an easy solution as it does not change the data the engine/game consumes. Just precompile the shaders, provide the SPIR-V code as the shader code and get the reflection data during runtime.

However, this approach is also rather inefficient as you will be extracting the reflection data a lot more often than you need to. And you will also be doing so at a very inconvenient time as it will just increase loading times. Additionally, beware that SPIRV-Cross requires exceptions to be enabled.

The best approach is to provide the reflection data together with the SPIR-V code as part of your assets. That way, you only extract the data once and have optimal loading times.

Graphics Pipelines

Graphics pipelines are a combination of shaders, vertex attribute descriptions and lots of settings for different parts of the rendering pipeline. What makes these pipelines difficult to deal with is the fact that their state also contains information about blending configuration, render target attachments and the viewport.

We have not found a good way to generically determine what pipelines are needed for a particular object before it is supposed to be rendered. This is difficult because, for example, new render targets that may need a completely new set of pipelines for all objects may be created at any time.

Our solution is to have our very own pipeline cache. The hash key for a given pipeline is all the data our renderer uses to generate a pipeline. Whenever an object is going to be rendered, we generate the required pipeline on the fly. This means an object may actually not show up for a short amount of time until the asynchronous pipeline generation has finished.

This approach can lead to no objects showing up at all in a new level for a short time. In order to avoid this, we try to pre-warm the cache by requesting those pipelines that we will probably need during the loading screen.

Vulkan also has the concept of a pipeline cache and even allows you to access its data to store it somewhere. You can use this stored data to pre-initialize the cache in a future run. In our case, compiling one pipeline took up to 50 milliseconds.

Unfortunately, there are various circumstances in which you cannot restore a pipeline cache from saved data. One example would be the driver making changes to the cache format. The cache format may also differ from vendor to vendor. As a result, it is sadly not feasible to ship a game with prebuilt pipelines (at least on Android).

Conclusion

Textures, shaders and pipelines are all assets you generally also encounter with other graphics APIs, but not necessarily in this form (pipelines), or they may have their own little differences and challenges (textures, shaders) with Vulkan.

We had to employ a few workarounds, like the usage of SPIRV-Cross at runtime and the caching of graphics pipelines, in order to get Vulkan to play nicely with our engine/tools setup. There certainly are better approaches that will result in better performance, but what we talked about here worked well enough for us.

In the next post, we will talk about a number of pitfalls we encountered on our Vulkan journey.