Sunday, October 16, 2011

Compressor, Part 2/2

What the hell we're we doing again? Oh yes, texture compression. In short, the main advantages are less storage space, less (video)memory space, faster loading/streaming, and if you're a lucky guy, a (little) performance boost cause less bandwidth is required. Reason enough to implement compressed texture formats... but how?

When you target OpenGL, you can encode pixelData into the DXT1, DXT3 or DXT5 format. So first (1.) encode, then (2.) store it in a file (a DDS for example), (3.) load the file later on, and finally (4.) send the (compressed) pixelData to the videocard. Sounds pretty complicated, but we have some nifty tools.

==========================================================
1. The Creation
==========================================================
Load a (raw) image, such as a bitmap or TGA file. Then encode it to a DXT format (or BC in case you target DirectX applications). So... we have to dig in compression algorithms now, right? Of course not. Don't reinvent the wheel, be lazy, and make good use of existing tools.

You can download an existing convertor tool or try to find a plugin for your favourite painting program. There are plugs for Photoshop, Paint Shop Pro, Gimp, Paint.NET... These plugins typically convert a common image format to a DDS file. You can skip step 1 and 2 of this tutorial now, peace of cake. Have a good look though. Not all convertors have equal quality and export options.

When my mobile research-lab was crunching on DDS, I had an old version of Paint Shop Pro though. Just ordered a new version (PSP X4), but no cool plugins at that time. I tried some other exporters, but they usually gave wrong output. Inverted RGB channels, flipped images, Lena changing into a guy, et cetera. Some claimed ATI has a good quality exporter, so I tried this instead: ATI Compress


Too bad, this is not a ready-to-use convertor, not even a command-line tool(I hate those, too lazy to type). It's a DLL you can include in your own programmed tools. Normally I would pass, but with the lack of good exporting tools so far, I gave it a try. And I got to say, it wasn't that hard at all (getting into C++ again took me more time). The ATI library is pretty straight forward. It has a function to compress raw image data, and some basic utilities to read/write DDS files.

1.- Load input image (bitmap, buttmap, tga, png, ...)
2.- Generate Mip-Maps (if you like)
3.- Let the ATI Compress library generate compressed pixelData (for each MipMap level)
4.- Store it as a DDS file (or your own custom format)

Mip-Maps ?!
In case you never heard of them, Mip-Maps are smaller ("blurred") variants of your texture. Pretty simple, half the size until you reach a 1x1 resolution. So a 256x256 image gets a 128, 64, 32, 16, 8, 4, 2 and 1 variant. But... why would you do that? Well, when looking a textured surface from a distance, one screen-pixel may have to sample from multiple texture-pixels. This can result in pixelated/blocky results, especially for textures with a high-frequency details or pattern such as black-white tiles. When you have mip-maps, the renderer will (automatically) pick a smaller, more blurry, variant to suppress this annoying artifact. A little downside is that mip-maps require extra image space (though the smaller resolution variants really don't take that much).

Sorry if you died from an epileptic attack.

OpenGL / DirectX have functions that generate mip-maps automatically for you. However, generating them takes time! So each time you load a texture that uses mip-mapping, you're wasting even more time with generating them. That's why some image formats (such as DDS) supports the storage of pre-fabricated mip-maps. So instead of rebuilding them each time again, you store smaller variants of the texture ready-to-use in the image file. In other words, it makes the loading times faster.

==========================================================
2. DDS File-Loader
==========================================================
Once your file has been generated somehow, you got to load it again in your app. I chose to use DDS files. Pretty obvious, cause this format supports compressed pixelData, storing prefab mip-maps, and even cubeMaps or layered (3D/Array) textures if you like. On top, it’s a common standard in Gameland nowadays. Monkeys see, monkeys do.

Luckily the DDS format is fairly easy. Mainly cause it doesn’t store its pixels in a wacky way. The array of pixels is stored exactly as OpenGL or DirectX expects it. This makes DDS fast to read, as you don’t have to swap bits or do other tricks before you load it to the GPU. It’s 1 on 1. But wait… D-D-S… stands for “Direct-Draw-Surace”… Isn’t that DirectX slang?? So we can’t use DDS in OpenGL apps?!

It is a DirectX thing indeed, and yep, you need a header file to get things working. But don’t worry. It’s not that your OpenGL program changes into a malformed Siamese twin with DirectX. Download the DirectX SDK, or if you are a Delphi user, you can use this: Clootie DX Pascal headers
Then include “DirectDraw.h / pas” in your program. Done.

// Delphi code.
function TEX_LoadFile_DDS( filename : string ) : TTexData;
var
ddsd :_DDSURFACEDESC2;
fileCode : array[0..3] of char;
factor : integer;
bufferSize : integer;
readBufferSize: integer;
pFile : THandle;
readBytes : Longword;
begin
{ Open file... Calling Powdered toast man }
pFile := CreateFile(PChar(filename), GENERIC_READ, FILE_SHARE_READ, nil, OPEN_EXISTING, 0, 0);
if (pFile = INVALID_HANDLE_VALUE) then begin
showMessage( 'DDS Load Error: Cannot open file ' + filename );
Exit;
end;

{ Verify if it is a true DDS file. Not made-in-China fake stuff }
ReadFile( pFile, fileCode, 4, ReadBytes, nil);
if (fileCode[0] + fileCode[1] + fileCode[2] <> 'DDS') then begin
showMessage( 'DDS Load Error: file is not a valid DDS file.'#13+filename );
CloseHandle(pFile);
exit;
end;

{ Read surface descriptor.
A struct that tells what we can expect in this file. }
ReadFile( pFile, ddsd, sizeof(ddsd), ReadBytes, nil );

case ddsd.ddpfPixelFormat.dwFourCC of
FOURCC_DXT1 :
begin
// DXT1's compression ratio is 8:1
result.outputFormat := GL_COMPRESSED_RGBA_S3TC_DXT1_EXT;
result.isCompressed := True;
factor := 2;
end;
FOURCC_DXT3 :
begin
// DXT3's compression ratio is 4:1
result.outputFormat := GL_COMPRESSED_RGBA_S3TC_DXT3_EXT;
result.isCompressed := True;
factor := 4;
end;
FOURCC_DXT5 :
begin
// DXT5's compression ratio is 4:1
result.outputFormat := GL_COMPRESSED_RGBA_S3TC_DXT5_EXT;
result.isCompressed := True;
factor := 4;
end;
else begin
{ Not compressed. Oh shit, didn't implement that! }
result.isCompressed := False;
showMessage( 'DDS Load Error: Uncompressed format not supported!'+#13 + filename );
CloseHandle(pFile);
exit;
end;
end; // case ddsd.ddpfPixelFormat.dwFourCC


{ How big will the buffer need to be to load all of the pixel data
including mip-maps? }
if( ddsd.dwLinearSize = 0 ) then
begin
showMessage( 'DDS Load Error: dwLinearSize is 0!'+#13 + filename );
CloseHandle(pFile);
exit;
end;
if( ddsd.dwMipMapCount > 1 ) then
bufferSize := ddsd.dwLinearSize * factor else
bufferSize := ddsd.dwLinearSize;

{ Allocate pixel buffer, then read the (compressed)
PixelData (containing 1 or more MipMap levels) from the file }
readBufferSize := bufferSize * sizeof(char); // Calc buffer-size
GetMem( result.pixels, readBufferSize ); // Allocate memory
ReadFile( pFile, result.data^ , readBufferSize, ReadBytes, nil);
CloseHandle(pFile); // Close file

{ More output info }
result.width := ddsd.dwWidth;
result.height := ddsd.dwHeight;
result.numMipMaps := ddsd.dwMipMapCount;

{ Do we have a fourth Alpha channel doc? }
if( ddsd.ddpfPixelFormat.dwFourCC = FOURCC_DXT1 ) then
result.components := 3 else
result.components := 4;
end; // TEX_LoadFile_DDS


Just a shot. Playing around with ingame contrast here...

==========================================================
3. Pumping to the video-card
==========================================================
The final, and probably also most easiest step, is loading the (compressed) pixelData to the video-card. No worries, it’s pretty much the same as generating any other texture. Start with the daily stuff:

glEnable( GL_TEXTURE_2D );
glGenTextures( 1, @resultHandle );
glBindTexture( GL_TEXTURE_2D, resultHandle);
// Texture settings. Using mipmapping here...
glTexParameteri(GL_TEXTURE_2D,, GL_TEXTURE_MIN_FILTER, GL_LINEAR_MIPMAP_LINEAR );
glTexParameteri(GL_TEXTURE_2D,, GL_TEXTURE_MAG_FILTER, GL_LINEAR );
glTexParameteri( GL_TEXTURE_2D, );

Next we send the pixelData to the Twilight zone. One little thing to keep in mind is that we might have loaded multiple mipmaps, all in the same array of pixelData. So for each mipmap level, calculate the bytesize and offset in the array.

if textureData.usingCompression then begin
if textureData.outputFormat = GL_COMPRESSED_RGBA_S3TC_DXT1_EXT then
nBlockSize := 8 else
nBlockSize := 16;
{ Size of mipmap level 0 (original size) }
nHeight := textureData.height;
nWidth := textureData.width;
nOffset := 0;

{ Send the compressed mipmap(s) data to the texture }
for i:=0 to data.numMipMaps-1 do begin
if nWidth = 0 then nWidth := 1;
if nHeight = 0 then nHeight := 1;

nSize := ((nWidth+3) div 4) * ((nHeight+3)div 4) * nBlockSize;
glCompressedTexImage2DARB( GL_TEXTURE_2D,
i, //mipmap level
data.outputformat, // DXT1,3,5
nWidth,
nHeight,
0,
nSize,
pointer( integer(data.data) + nOffset)
);
nOffset := nOffset + nSize; // Offset in pixel buffer next time
// Half the image size for the next mip-map level...
nWidth := (nWidth div 2);
nHeight := (nHeight div 2);
end; // for i
end else ...

Texture quality
Last, here's a little Kung-fu trick. Probably you noticed the "Texture Quality" setting in most games. Chose between "Godlike, medium or fucked-up". Obviously, computers with lower-end hardware and/or limited video-memory should pick a lower setting. And you want this option too, don't you? Pretty easy. Just skip the first mipmap level(s) in the loop. This way OpenGL will only deal with smaller textures, as the higher level(s) get disposed again. That's all you need to know for now, Daniel San.

No comments:

Post a Comment