Compressor, Part 1/2

In case you are a programmer or texture-artist, you probably have heard about DDS files or texture compression. But if you are like me, you may have skipped all those articles. Pfff, compression, who needs that? Bitmaps and TGA files work for me, Pixel-data is pixel-data right? Besides, I've been taught that compression makes the quality worse, and reduces the performance because in the end, your pixels still have to be decompressed to a raw format. So what's the point?

Well forgive my ignorance. Guess I'm still traumatized because 12 years ago, I couldn't figure out why my sprites had jaggy pink edges when using JPEG files. Oh yes, the compression did that. But although the quality / performance arguments are valid, texture compression actually gives some nice possibilities. In this post, I'll write about using DXT compressed textures in your OpenGL program, and the DDS file format that can store this format.


1. For beginners: (de)compression
------------------------------------------------------------------------
For starters, as the name sais, compression is a technique to store (large) buffers of data with some tricks to reduce the size. When it comes to images, we usually have a big array of bytes. In a classic RGB (24 bit) image, each pixel takes 3 bytes. 8 bit for red, 8 bit for green, 8 bit for blue. Some images also have an 8-bit alpha channel, which makes the total size of a pixel 32 bit. Calculating the total buffer size is easy:
.....width * height * pixelSize. 512 x 512 x RGBA8(32 bit) = 1 MB

Cool, but 1 megabyte for a pretty small texture is huge (1 floppydisk!). Older photo camera’s would have their SD cards full after just a few clicks. So, that's why JPEG is often used. JPEG uses Huffman encoding, which is based on the probability a certain piece of data (pixelcolor) occurs. If we only have 20 different colors in an image, we can index the pixelcolors with a 5-bit (=32 possibilities) code. Further compression can be done by giving frequent colors a short bitcode, while rare colors get a longer code. Well, for a way better description, check this.

Smart huh? However, compressing the image has three main disadvantages. First there is usually quality loss. You probably have seen blurry blocks in JPEG images, especially when the color contrast is low. To put it simple, most compressed images NEVER reach the quality of a raw format (such as BMP or uncompressed TGA / PNG). Then again, if the resolution is big enough, compression settings are ok, and/or the small details don't really matter anyway, the loss is acceptable.

Second problem is the decompression-time. It simply takes some more time to convert the cryptic pixelmess to a raw format. Your video-card needs to produce raw (RGB 8-bit) pixeldata first, before the monitor can show it. Compare it to translating a Chinese movie to English. First a translator has to write down the English subtitles or do voice acting before it can be shown on TV. This process takes some time.

A third little problem is the compression itself. Just like decoding, encoding takes some extra effort as well. Luckily computers are fast these days, so you won't really notice. However... it's not fast enough to do many textures on the fly while rendering a game. This is why it usually happens "offline":

1.- Create your image (photograph / paint / capture raw data)
2.- Compress it in whatever file format
3.- Load in your application
4.- Decompress it back to raw-data
5.- Use (render) it.

Usually some quality loss is acceptable.


2. Compression & realtime Graphics
------------------------------------------------------------------------
So far, the only main advantage of using compressed images, is the reduced space on disk. You can't just load a JPEG and send it to the video-card texture memory right away. Nope, first you'll have to decode the data, then send a raw buffer of colors to the video-mem. So in the end you are still using the same amount of memory, it took extra time to decode, AND you lost quality. In other words, screw JPEG or any other compressed image format for graphics.

But wait. Things have changed a bit. Video-card memory has grown a lot last 10 years. From 32 MB to a gig or more. Unfortunately, games grow even faster. We want ultra-high res textures, and LOT's of them. Using all those textures at the same time requires quite a lot memory space, and bandwidth. Bandwidth? Imagine 10.000 hungry fat guys. You need 100 mcDonalds trucks to transport hamburgers. If all those trucks need to travel through a tunnel at the same time you get traffic-jams. Wouldn't it be nice if a single truck can carry more hamburgers, ending up with less trucks?


So, the graphic masters came up with compression algorithms specially designed for video-cards. I'm a newbie here as well, so I can't tell you the fine details, but you may have heard about DXT1, DXT3, DXT5, S3TC, BC1..7. Each of them is a compression method. Picking one is a trade-of between quality and space, and it also depends on what kind of image you want to compress (a normalMap, a simple blurry texture, a grayscale image, transparency yes/no). The compression ratio can be up to 1:6, which means a 1MB texture becomes ~170 kb.

The way the pixels are trashed together, is compatible with OpenGL / DirectX. That means you don't have to decompress the pixeldata first, before sending it to the videocard. No, you can send the packed buffer right away. That means the video-memory will be spared as well. Decompression happens on the fly by the hardware. I'm not sure if there is no performance penalty at all, but it seems the hardware can read compressed data just as fast as raw data. In fact...
you can even boost the performance! Remember the mcDonalds trucks? Since you need to transport less data on the buses, there will be less bandwidth issues as well. Now you probably won't notice this is you were only using a few textures anyway, but complex scenery such as a FPS game can benefit... Up to 20% according to some!

So far only 10 different textures are used in this snapshot. Or well, actually 20 as most objects also use a second or third image (normal/emissive/specular/height Map). Most textures have a resolution of 512x512 or 1024x1024, RGBA. So in total, there would be around (1024*1024 * 4(rgba) * 20(textures) = 80 MB of texture data pumped around every cycle (mip-maps, shadowMaps and deferred buffers not included). With compression, we can reduce this to ~20 MB.

Another cool detail is the loading time. Since you don’t need to convert stuff, and your harddisk or Blu-ray has to read less bytes, streaming goes faster. Who doesn’t like fast loading times? Tower22 streams maps and textures in the background while playing, so faster streaming is certainly nice.

Advantages
+ Less space required on disk / CD-Rom / Floppy disk / Blu-ray / Tape
+ Smaller files are loaded / streamed faster as well
+ Pre-build mipMaps in DDS files make the loading even faster.
+ Less space required in the video memory
+ Less bandwidth required. *Could* be a performance win.
+ Less quality maybe... but you can use bigger resolutions as well!
Disadvantages
- Quality loss. In some cases, compression is not worth it.
- Extra offline steps for the artists to produce compressed (DDS) files.

About the quality loss, this depends a bit. Matt(MJP) gave me some tips:

* Chose the proper compression methods
......- DXT1: Lower quality, higher compression, 1-bit alpha only(on/off)
......- DXT3: Better quality RGB, 4-bits for the alpha. Suitable for transparent ...... textures like foliage or a metal fence/grate.
......- DXT5: Suitable for images with smooth alpha channels or for normalMaps. Compress R and G, reconstruct B in shader.

* Use the right compression tool (nVidia, ATI, Microsoft, ...). Some do a better job than others, so carefully pick your weapons.

* You could pick a bigger resolution to compensate the quality loss, and still use less space in the end in some cases.

* Keep the option to use uncompressed textures, in case the quality sucks too much hairy balls.

Okidoki, but how in the name of SaintCrap can we make use of it? Consider three steps:
1.- Create a file with compressed image data... DDS files!
2.- Load the file in your program
3.- Create a (OpenGL/DirectX) texture using the compressed data

Next time, I'll tell you how to create DDS files, how to load them, and how to create an OpenGL texture using an (DXT) compressed image. Now it's time for beer.