Let's continue this "beginner graphics tour". Again, if you made shaders or a lot of game-textures before, you can skip. But keep hanging if you want to know more about how (basic) lighting works in games, and what kind of textures can be used to improve detail.
Last time we discussed how the classic "texture" is applied on objects. Basically it tells which colors appear on an object, wall, or whatsoever. To add a bit of realism, that color can be multiplied with the color from lightsources (sun, flashlight, lamps, ...) that affect this pixel. Being affected means the lightrays can "see"/reach this pixel directly or indirectly (via a bounce on another surface). A very crude lighting formula would go:
...resultColor = texturePixelColor * (light1Color + light2Color + ... + lightNcolor)
Thus we sum up all incoming light, and multiply it with the surface color (usually fetched from a texture). If there are no lights affecting the surface, the result appears black. Or we can use an ambient color:
...resultColor = texturePixelColor * (light1Color + light2Color + .... + lightNcolor + ambientColor)
This formula is simpl, but checking if & how much a light affects a surface is a whole different story though. It depends on distance, angles, surroundings, obstacles casting shadows, and so on. That's why old games usually had all this complex stuff pre-calculated in a so called lightMap, just another texture being wrapped over the scene and multiplied with the surface textures. The big problem of having it all pre-computed is that we can't change a lighgt. So, old games had static lights (= not moving or changing colors), no shadows on moving objects, and/or only very few physically incorrect lights that would be updated realtime.
Shadows? Anyone? The little lighting around the lamps are pre-baked in low resolution lightMaps. Halflife 1 was running on an enhanced Quake2 engine btw.
When shaders introduced themselves in the early 21th century, we programmers (and artists) suddenly had to approach the lighting on a more physical correct way. Shaders by the way are small programs running on the videocard that calculate where to place a vertex, or how a certain pixel on the screen should look(color/value/transparency). With more memory available and videocards being able to combine multiple textures in a flexible programmable way, a lot of new techniques popped up. Well, not always new really. The famous "bumpMapping" for example was already invented 1978 or something. But computers simply didn't have the power to do anything with it (realtime), although special 3D rendering software may have used it already. Think about Toy Story (1995).
Anyway, it caused several new textures to be made by the artists, and also forced us to call our textures in a bit more physical-correct way. The good old "texture" on objects/walls/floors suddenly became the "diffuseTexture", or "albedoTexture". Or "diffuseMap" / "albedoMap". Why those difficult words? Because the pixels inside those textures actually represent how the diffuse light should reflect, or the "albedo property" of your material. Don't worry, I'll keep it simple. Just try to remember your physics teacher telling about light scattering. No?
The art of lighting
When lightrays (photons) fall on a surface, a couple of things happen:
- Absorption:In other words, photons bounce of a surface, and if they reach your eye, you can see it. The amount of photons you receive depends on your position, the position and intensity of the lightsource(s), and the albedo + specular properties of the material you are looking at. The cones in your eyes convert that to an image, or audio waves if you are tripping on something. So. your texture -now called albedo or diffuseMap- tells the shader how much diffuse-light(and in which color) it should bounce to the camera. Based on that information, the shader calculates the resulting pixel color.
A part of the photons gets absorbed (black materials absorb most).
- Specular reflection:
Another portion reflects on the surface "normal". In the pic, the normal is pointing upwards. This is called "specular lighting". The "specularity property" tells how much a material reflects. Typically hard polished surfaces like metals, plastics or mirrors have a relative high specularity.
- Diffuse scattering:
The rest gets scattered in all directions over the surface hemisphere. This is called "diffuse lighting". The "albedo property" of a material tells how much gets scattered. So high absorbing(dark) surfaces have a low albedo. The reason why it scatters in all directions is because the roughness of a material (on a microscopic level).
… diffuseLight = sum( diffuseFromAllLights ) * material.albedoProperty; specularLight = sum( specularFromAllLights ) * material.specularProperty; pixelColor = diffuseLight + specularLight + material.Emissive // Notice that indirect light (GI) is also part of the diffuse- and specular light // However, since computing all these components correctly, GI is usually added // afterwards. Could be a simple color, could be an approximation, could be anything… pixelColor += areaAmbientColor * material.albedoProperty * pixelOcclusion;This stuff already happened in old (90ies era) games, although on a less accurate level (and via fixed OpenGL/DirectX functions instead of shaders). One of the main differences was that old graphics-engines didn't calculate the diffuse/specular lighting per screen pixel, but per vertex. Why? Because its less work, that's why. Back then, the screen resolution was maybe 640 x 480 = 307.200 pixels. That means the videocard (or still CPU back then) had to calculate the diffuse/specular for at least 307.200 pixels. Likely more, in case objects would overlap each other the same pixel would be overwritten and required multiple calculations. But what if we do it per vertex? A monster model had about 700 vertices maybe, and the room around you even less. So a room filled with a few monsters and the gun in your hands had a few thousand vertices in total. That is way less than 307.200 pixels. So, the render-engine would calculate the lighting per vertex, and interpolate the results over a triangle between its 3 corner-vertices. Interpolation is a lot cheaper than doing light math.
DiffuseLighting & DiffuseMaps
--------------------------------
I already pointed it out more or less, but let's see if we get it (you can skip the code if you are not interested in the programming part btw). Pixelcolors are basically a composition of diffuse, specular, ambient and emissive light. At least, if your aim is to render "physically correct". Notice that you don't always have to calculate all 4 of them. In old games materials rarely had specular attributes (too difficult to render nicely), ambientLight was a simple fake, and very little materials are emissive (= emitting light themselves). Also in real life, most light you receive is the result of diffuse lighting, so focus on that first.
We (can) use a diffuse- or albedo image to tell the diffuse reflectance color per pixel. The actual lighting math is a bit more complicated, although most games reduced it to a relative simple formula, called "Lambertian lighting":
// Compute how much diffuse light 1 specific lamp generates
// 1. First calculate the distance and direction vector from the pixel towards
// the lamp.
float3 pixToLightVector = lamp.position - pixel.position;
float pixToLightDistance = length( pixToLightVector );
pixToLightVector = normalize( pixToLightVector ); // Make it a direction vector
// 2. Do Cosine/Lambert lighting with a dot-product.
float diffuseTerm = dot( pixel.normal, pixToLightVector );
diffuseTerm = max( 0, diffuseTerm ); // Value below0 will get clamped to 0
// 3. Calculate attenuation
// Light fades out after a distances (because lightrays scatter)
// This is just a simple linear fall-off function. Use sqrt/pow/log to make curves
float attenuation = 1 - min( 1, pixToLightDistance / light.falloffDistance );
// 4. Shadows (not standard part of Lambert lighting formula!!)
// At this point (in this shader) you are not aware if there are obstacles
// between the pixel and the lamp. If you want shadows, you need additional
// info like shadowMaps to compute wether the lamp affects the pixel or not.
float shaded = myFunction_InShadow( ... ); // return 0..1 value
// 5. Compose Result
lamp1Diffuse = (diffuseTerm * attenuation * shaded) * lamp1.color;
// ... Do the same thing for other lights, sum the results, and multiply
// with your your albedo/diffuse Texture
Wanting to learn programming this? Print this picture and hang it on the toilet so you can think about it each time when pooping. works like a charm.
You don't have to understand the code really. Just realise that your pixels get litten by a lamp if
A: The angle between the pixel and lamp is smaller than 90 degrees (dot)
B: The distance between pixel and lamp is not too big (attenuation)
C: Optional, there are no obstacles between the pixel and the lamp (casting shadows)
One last note about diffuseMaps. If you compare nowadays textures to older game textures, you may notice the modern textures lack shading and bright highlights. The texture is more opaque... because it's a diffuseTexture. In old games, they didn't have techniques/power to compute shadows and specular highlights in realtime, so instead they just drawed ("pre-baked") that into the textures to add some fake relief. Nowadays many of those effects are calculated at the fly, so we don't have to draw them anymore in the textures. To some extent, it makes drawing diffuseMaps easier these days. I’ll explain in the upcoming parts.
SpecularLighting & SpecularMaps
--------------------------------
Take a look back at one of the first pictures. Diffuse light scatters in all directions and therefore is view-independent. Specular lighting on the other hand is a more concentrated beam that reflects on the surface. If your eye/camera happens to be at the right position, you catch specular light, otherwise not. Specular lighting is typically used to make polished/hard materials “shiny”.
Again, this is one of those techniques that came alive along with per-pixel shaders. Specular lighting has been around for a long time (a basic function of OpenGL), but not used often in pre-2000 games. A common 3D scene didn’t (and still doesn’t) had that much triangles/vertices. Since older lighting methods were using interpolation between vertices, there is a big lack of accuracy. Acceptable for diffuseLighting maybe, but not for specular highlights that are only applied very locally. With vertex-lighting, specular would appear as ugly dots. So instead, games used fake reflection images with some creative UV mapping (“cube or sphere mapping”) to achieve effects like a chrome revolver. Nowadays, the light is calculated per pixel though, so that means much sharper results.
If you looke carefully, you can literally count the vertices on the left picture. Also notice the diffuse light (see bottom) isn't as smoothy, although less noticable.
That gave ideas to the render-engine nerds... So we calculate light per pixel now, and we can vary the diffuse results by using a diffuse/albedo texture... Then why not make an extra texture that contains the specularity per pixel, rather than defining a single specular value for the entire object/texture instead? It makes sense, just look at a shiny object in your house, a wood table for example. The specularity isn't the same for the entire table. Damaged or dirty parts (food / dust / fingerprints / grease) will alter the specularity. Don't believe me? Put a flashlight on the end of the table, and place your head at the other hand of the table. Now smear a thin layer of grease on the middle of the table. Not only does it reduce the specurity compared to polished wood, it also makes the specular appear more glossy/blurry, as the microstructure is more rough. You can only see this btw if the flashlight rays exactly bounce into your eyes. If you stand above the table, the grease may become nearly invisible. That's why we call specular lighting "view dependant".
Dirty materials, or objects composed of multiple materials (old fashioned wood TV with glass screen and fabric control panel) should have multiple specular values as well. One effective way is to use a texture for that, so we can vary the specularity per pixel. Same idea as varying the diffuse colors in an ordinary texture (albedo / diffuseMap). A common way to give your materials a per-pixel specularity property, is to draw a grayscaled image where bright pixels indicate a high specularity, and dark pixels a low value. You can pass both textures to to your shader to compute results:
float3 pixToLightVector = lamp.position - pixel.position;
float pixToLightDistance = length( pixToLightVector );
pixToLightVector = normalize( pixToLightVector );
float3 pixToEyeVector = normalize( camera.Position - pixel.position );
// Do Cosine/Lambert lighting
float diffuseTerm = dot( pixel.normal, pixToLightVector );
diffuseTerm = max( 0, diffuseTerm );
float3 diffuseLight = diffuseTerm * lamp.color;
// Do Blinn Specular lighting
float3 halfAngle = normalize( pixToLightVector + pixToEyeVector );
float blinn = max( 0, dot( halfAngle, pixel.normal.xyz ) );
float specularTerm = pow( blinn , shininessPowerFactor );
float3 specularLight= specularTerm * lamp.color; // eventually use a 2nd color
...apply shadow / attenuation...
// Fetch the texture data
pixel.albedo = readTexture( diffuseMap, uvCoordinates );
pixel.specularity = readTexture( specularMap, uvCoordinates );
diffuseLight *= pixel.albedo;
specularLight *= pixel.specularity;
Notice that this is not truly physical correct lighting, it's an approximation suitable for realtime rendering. Also notice that the method above is just one way to do it. There are other specular models out there, as well as alternate diffuse models. Anyway, to save some memory and to make this code a bit faster, we can skip a "readTexture" by combining the diffuse-and specularMap. Since the specularity is often stored as a grayscale value, it can be placed in the diffuseMap alpha channel (resulting in a 32-bit texture instead of 24bit). The code would then be
// Get pixel surface properties from vertices and 2 textures float4 rgbaValue = readTexture( diffuseSpecMap, uvCoordinates ) pixel.albedo = rgbaValue.rgb; pixel.specularity = rgbaValue.a;
However, some engines or materials chose to use a colored specularMap, and eventually use its alpha channel to store additional "gloss" or "shininess" data. This is a coeffcient that tells how sharp the specular highlightwill look. A low value will make the specular spread wider over the surface (more glossy), while high values make the specular spot very sharp and narrow. Usually hard polished materials such as plastic or glass have a high value compared to more diffuse materials.
spec = power( reflectedSpecularFactor, shininessCoefficient )
EmissiveLighting & EmissiveMaps
--------------------------------
This one is pretty easy to explain. Some materials emit light themselves. Think about LEDs, neon letters, or computer displays. Or radioactive zombies. When using diffuse- and specular light only, pixels that aren't affected by any light will turn black. You don't want your TV appear black in a dark room though, so one easy trick is to apply an EmissiveMap. This is just another texture that contains the colors that are emitting light themselves. Just add that value to the result, and done. The shader code from above would be expanded with
resultRGBcolor = blabla diffuse blabla specular blabla ambientNotice that emissiveMapping is just another fake though. With this, your TV screen will appear even in dark rooms. And it may also get reflected and cause a "bloom"(blur) around the screen in modern engines. BUT, it does not actually emit light to other surfaces! It only affects the pixels you apply this texture on, not the surrounding environment... unless you throw some complex code against it, but most games just place a simple lightsource inside the TV, or ignore it completely.
resultRGBcolor = resultRGBcolor + textureRead( emissiveMap, uvCoordinates );
The charming man on the TV and the lamp in the background use emissive textures. However, the TV does not really cast light into the scene (other than the blurry reflection on the ground).
I could go on a bit longer, but let’s take a break. I remember when I had to read this kind of stuff ~7 years ago… Or well, probably I didn’t really read it. My head always starts to hurt if I see more than 3 formula’s with Sigma and 10th order stuff. But don’t worry, just go out and play, you’ll learn it sooner or later, and documents like this will confirm (or correct) your findings. The next and last part will reveal the famous “bumpMaping” technique, plus describes a few more texture techniques that can be used in modern games. Ciao!
Interesting! Also i love those graphics!
ReplyDeleteMy pleasure :)
ReplyDeleteReally a great addition. I have read this marvelous post. Thanks for sharing information about it. I really like that. Thanks so lot for your convence.
ReplyDeleteWall Lights