DXT Texture Compression in 2018

[Originally from http://onemanmmo.com where I'm building the first seamless open-world RTS.]

Two weeks ago I knew exactly zero about DXT. I was surprised to find a lack of modern resources on the topic and spent two days researching to get started. It's been ten months since I've written anything really technical and I've been missing that.

The goal is to reduce the amount of GPU memory required for textures. The reason for that is that I am going to be drastically increasing the amount of textures in the game in the coming weeks as I do a major overhaul to Miranda's rendering and art, and according to Steam, 14.48% of gamers still have only 1GB of VRAM. The two weeks adding support for DXT texture compression are for those people. For the 2.76% of players with less than 1GB of VRAM, Miranda isn't going to work any longer after the next update.

Up until today, Miranda only supported run-time compression (using GL_COMPRESSED_*) which it turns out isn't really a solution since it can take seconds per-texture for the GPU to compress them on load. The GPU VRAM savings of run-time compression is alright, but not great.
 

[Miranda with DXT compressed textures - it looks the same!]

DXT

There are three DXT codecs we're interested in: DXT1, DXT3 and DXT5. All of these are block oriented codecs that work on 4x4 blocks of pixels. If your texture isn't a multiple of four in size, the size is rounded up. Each DXT1 block compresses down to 64 bits (6:1). Each DXT3 and DXT5 block compresses to 128 bits (4:1). DXT1 works on RGB data. DXT3 & DXT5 also provide an alpha channel (only 4 bits), however DXT5 is considered the better of the two. Technically DXT1 supports 1 bit of alpha, but I'm not using that.
 

Newer Codecs

There are a number of newer codecs, ETC1 & ETC2 have been adopted by Khronos group for use in OpenGL. There are also BC4-7 (DXT1/3/5 are the same as BC1-3.) The problem with these newer codecs is that on PC at least, there apparently isn't much hardware support for them, which means that when they are loaded they are transcoded into a format that the GPU can handle. If you're working on mobile, you'll probably care about maximal compression so you'll want to investigate what codecs your handsets support.
 

DDS Files

The preferred format for DXT encoded bitmaps is the DDS file, which it turns out is a really straightforward file format to load. Here's a DDS loader source example from NVidia. The only tricky part of writing the DDS loader was the FOURCC's which are in the reverse order on Intel. Microsoft's DDS Programming Guide was really helpful for making sure I didn't miss any implementation details by working from a source example.
 

MIP Maps

When rendering textures on models far from the camera, MIP Maps improve performance by using pre-scaled-down versions of textures. A bitmap with MIP maps takes one and a third times as much RAM. Previously Miranda generated mipmaps on a few textures at load time with glGenerateMipmap. The MIP maps produced this way are alright, but not optimal, and they take time to generate. DDS files support MIP maps natively.
 

DXT Libraries

Originally I thought I would use a library and do DXT compression in a tool myself. I looked at a number of the available libraries.

TextureImport

Building my own tool to do DXT compression and generate MIP maps seemed like an unnecessary extra effort once I had seen Compressonator's command line utility. For a while I thought I'd just have a batch file to run Compressonator and update that every time I added a new texture to the game. Then I looked at the number of textures I already had. I would have to figure out if they needed MIP maps, and the optimal DXT compression setting. I realized I could probably make a tool faster than I could write a batch file and select good settings for a couple hundred textures.

TextureImport ended up pretty cool. It reads every material in the game, figures out all the textures they use, then determines if they need MIP maps and if they can be DXT compressed based on the existing material definitions or if they need to remain uncompressed. It also checks the time on the material and input bitmaps, as well as any existing output files to determine if files need to be rebuilt. It then runs Compressonator to do the actual compression. When I add a new material into the game, the texture pipeline is now totally automated which is pretty handy for a single developer. The tool itself took just two days. It also discovered some no longer used materials, and removed all unused bitmaps from the game.
 

OpenGL Changes

The changes to support DXT files in the game were very minor. Anywhere I used glTexImage* I added support for glCompressedTexImage* and used the correct data type (GL_COMPRESSED_RGBA_S3TC_DXT1_EXT etc.) The only other change was to check for the GL_EXT_texture_compression_s3tc extension which is now required to run the game (it is one of the best supported OpenGL extensions.)
 

DDS Don't Do Gamma

Shortly after I had all of this working, I realized that the lighting on the terrain was wrong. This quickly led me to remember that my normal maps are linear (gamma 1.0) unlike most of Miranda's other textures which are sRGB (gamma 2.2.) Well, DDS files don't really handle Gamma. After a few hours digging through the source code of various DDS compression tools and much googling, I learned that newer DX10 formatted DDS files have some limited linear support, but nothing that would help me. I had modified my DDS routines to support DX10 formatted DDS files and added a DDS writer, then I discovered that nobody seems to support DX10 DDS files - so debugging them was a huge pain. I realized I was wasting my time trying to get DDS working, and went ahead and created the new Lair Texture format (.ltx - and yes I know that's the same extension as LaTex, but just try to find an unused file extension nowadays!) That worked really well. .ltx files are perfectly formatted for the game, support gamma, load super-fast, and with compression they brought the texture disk size down even further. The only downside is that Windows Preview can no longer display them. I'm still using Compressonator to do DXT compression, but as soon as it generates a new DDS file, I read that in and write out a new .ltx file with the correct gamma setting.
 

Results

The game looks more or less exactly like it did before, but now the textures take only 37MB of disk space instead of 252MB and my plans for upgrading Miranda's art are now possible. I added support for all the fancy anisotropic texture filtering modes to give players with powerful graphics cards the option of better looking graphics. I also got a big GPU memory savings by changing any bitmaps which didn't use alpha to be 24-bit. 24-bit bitmap support came late to the Lair Engine so most textures were loaded as 32-bit even though they didn't use alpha. The other big effort on this upgrade was modifying the Lair Engine's internal bitmap code to support DXT data, 3D bitmaps and MIP Maps.