CSI Profiler

Trying to complete all the items on the list with "things to-do before I die". I meant the list for the movie. But as usual with programming, such lists only grow instead of shrink. Let’s see... "Spawn object X after trigger Y". O, that’s simple. Wait... a few script functions are still missing. Graphical glitch here, graphical glitch there. Found a few new bugs. Don't like the spawn effect, want to morph it from object A to B...

And before you know it you spend another few evenings on what seemed to be a simple point. Well better fix them thoroughly instead of rushing and hacking in features. Making an engine for your game is like collecting stamps, you’ll keep adding items in that book.

Strange enough, I always ignored a particular nasty issue: performance. I don't have to tell you that a game should run fast enough to be playable. I always try to stay above the critical 30-frames-per-second zone. I once learned that our eyes can see about 30 frames per second, so the game framerate should definitely not drop below. I maintained that promise for quite a long time on my laptop. Not the fastest system ever built, but not slow either:
-Toshiba Qosmio 32 bit (Yes, the ugly flaming space station model with hyperdrive)
-4gig RAM, dual core
-and more important, nVidia 9800 M GTS

Blabla. It can run Crysis Warhead just about fast enough most of the time, that's what matters. Crysis is not really the same as my mostly indoor engine (ha, I wish), but it's an indication that beautiful things can be done on this computer, so no excuses. However, I don't care that much about performance yet. For one reason, if I ever finish this game, we'll be playing on a GeForce-27, Octa-Core, organic RAM computer with a stunning 60 inch floppy drive.

Figured this bed needed some clean sheets.

Fine and all, but the average FPS of my game is currently ~21. That's going to be a crappy ride when FRAPS is also running to capture that movie. A few months ago I upgraded the Cg(shader) DLL's from version 1.5 to 2.2. I needed it to realize Cascaded Lights, but as a reward my framerate dropped from ~50 to ~28. I searched like mad, asked the Cg forums, but couldn't find a cause. Well, I kind of ignored it, and in the meanwhile more effects plus more complex scenery were made as well.

Ignoring problems in software is always a bad idea. The longer you wait, the more difficult it gets to fix it. And worse, tracing problems is about the most boring part of programming if you ask me. After all, this is still a hobby, supposed to be fun you know… I’m not coming home after work to torture myself with pointer errors made by bad multi-threading (did that once a few weeks, and it sucked). Anyway, this engine has thousands of routines that influence the performance. Now where to look?

nVidia has several tools to monitor your graphical flow. However, I'm starting to pay the price for using older techniques as well. Delphi 7, and an older version of OpenGL. Not that I try to avoid newer techniques, but I simply don't want to start all over again. A next game/engine may be DirectX & C++, but let's try to finish something. Just for once.

So I made a "profiler" that simply checks the elapsed microseconds to perform subtasks. For example, how long does it take to render the SSAO effect? How many microsecs did it take to play the fart sound? Is the CPU having problems with throwing a wheelchair down the stairs? This allows to find some of the chokepoints between all those many routines.

The Hourglass model is used to measure the framerate. Got to be patient.

It still doesn’t really finds all problems though. It seems that the GPU doesn’t run 100% synchronized with the CPU. That means your program will proceed, thinking that rendering object X is already done, while in reality the video card is still busy. Nevertheless, it brought me to a piece of the HDR pipeline.

Coded years ago, I almost forgot that I was using glReadPixels to fetch the current screen luminance. A required component to do Tone Mapping (= adjusting the eye to the current brightness). This operation isn’t that slow on itself, but it interrupts the rendering process. Imagine driving 120 on the highway and suddenly you’ll have to hit the brakes because a police officer wants to test your alcohol BAC (or the screen luminance for that matter).

It came down to changing a few things so that the luminance would be passed by a tiny texture instead of letting the CPU intervene. Thus fully driven GPU HDR. While dealing with HDR, I decided to face another old problem as well; the pixel colors that came out the Tone Mapper.

The main colors here are white, blue and green... But I used to get bluish all over the place.

After all that scaling, stretching, messing, tossing and mixing, the overall colors would look a little bit washed out. Blue wasn't blue, green wasn't green, etcetera. Nice to make a particular atmosphere (bluish for example), but if I ask for a red sink, then I want a red sink dammit. Colorizing the whole scene is already possible as an post effect anyway.

After tweaking the Tone Mapping formula a little bit, I think the colors are better now, although an overall change like this one always messes up all the fine tuning you did before on the scenery. Really, some tiny little changes in the fog, brightness or colors can make it Hot or Not. Like I said, my list of things-to-do just increased a little bit again.

Ernst knows what's good for ya

Sigh. Another idiot that wants (violent) games to be forbidden. This time the fame goes to our own minister Ernst Hirsch Ballin. We Dutchman like to be known as progressive, open minded people. Legally smoke weed and space out with the hooters in Amsterdam, gay-power to the max, abortion, euthanasia, stuff yourself with mushrooms, free from religion, and so on. We used to laugh on America when we heard some petty discussions (drugs are bad mmkay, weapon shops ok, Michael Moore stuff). But while America is finally trying to scrap that famous f*BEEP*cking BEEP from their televisions, we start to forbid more and more. And for the wrong reasons.

To be more specific, our minister Ernst wants to fully forbid violent games because the 18+ labels do not work. Little kids still play Grand Theft Auto, and that is horribly wrong. Of course, everyone protests when his (bad) habits are questioned. Ask someone to quit smoking, drinking or playing bloody games, and you’ll get a defensive answer. So, are games really 100% innocent? I don’t think so. But guys like good old Ernst, Hillary Clinton or that moron Jack Thompson have to do their homework before yelling stupidity.

To clarify the mindsets of these people; Hillary was “shocked” by the Hot Coffee mod. That’s right, kids that are 16 or older still shouldn’t see 2 low-poly puppets having sex (with their underpants still on). Ernst tries to forbid anything that might hurt the Lords feelings, and instructed to arrest a cartoonist that made a silly joke (freedom of speech). And Jack stated that even The Sims is an adult game because little children can hack(year right) the game. I’ve seen the naked Sims indeed. They look like… a naked Barbie of my daughter. I’d better buy a chastity for Barbie before mr. Teddy Bear and Tyrannosaurus are on top. But ok, they still have a point.

Off topic (or on-topic, as this is a development blog), this week I was busy with the HUD. When picking up something, scripts can show dialog forms, play sounds, pause the game, or Sepia/blur the background. The challenge was to make it fully adjustable. If I want another dialog in the future, it should be possible without even touching the engine code.

I won’t play or watch something abusive when my little daughter is around. The reason is obvious, I don’t want her to have nightmares. And more important, she should not start thinking that beating up grandma’s, stealing cars, swearing or decapitating zombies is normal behavior. So, in essence these people are right when stating that (young) children should be protected. Though I’ll have to admit that I saw Jaws when I was about 5 years old, played Doom and Crusader No Remorse when I was 11/12. Never had any problems with aggression though. Same thing for most other people I know. My parents didn’t close my eyes when boobs showed on the TV, neither forbid me to play Carmageddon. All they did was simply keeping a close eye on me, and intervene when I would start showing “bad signals”.

As many other parents, they handled blood, sex, drugs and rock&roll on a stolid way. The right way. You all know what happens if you explicitly forbid girlfriends or alcohol; the more reason to try it out! Example, here in Holland where softdrugs are legal, we most probably use far less than our neighbors where drugs are still illegal. Then again, parents shouldn’t be laconic on the matter either. Children have to know what’s right and wrong. Same thing with games & internet. Pull out that internet cable if those kids are 24/7 online, simple as that. Like with most temptations in life, you have to moderate, taking the right doses.

Forbidding violent games is unnecessary, hypocrite, and on top, it won’t solve the real problems. As often with these subjective “researches”, causes and consequences are swapped. Many banana’s grow in the jungle because there are monkeys. I bet a Jack Thompson never even played a game, except Tiddly Winks maybe. In other words, conclusions made on a very wrong foundation. Everyone is free to have an opinion, but guys like Ernst or Hillary Clinton actually have the power to make policies, and that makes them dangerous(ly stupid). Well Ernst, I doubt if you read this filthy blog, but… Forbidding is:

Unnecessary:
Every incident inspired or otherwise related with games is one too many. But honestly, how many crimes came out of a game? We had a few psychopaths such as the Columbine boys, or wacko’s that kill each other because their virtual Goblin sword got stolen in World of Warcraft. But compare those numbers to all other criminals, rapers, Ted Bundy’s, pedophiles and other scum. You defineletly don’t need a game to get berserk. Typically, the most violent regions are usually places where they lack Wii’s and Xboxes (Somalia, Middle East, Ghetto’s, …). Also our own violent history moments such as WWII, throwing tar over each other, or crusades were written in gameless times. I doubt if Stalin, William Wallace or Alexander The Great had a Commodore 64. In short, gaming is not a significant causer of violent behavior. Maybe it makes kids anti-social, when playing too much. But that is still far away from hurting people.

Do not worry. This weapon fires confetti, no violence was used in my game

Hypocrite:
I’m not saying that games do not have any influence at all. But the same things can be said about movies, books, gangster rap or heavy metal. How about alcohol or football? These two things actually do cause quite a lot of violence, vandalism and family drama’s. Should we scrap them? Of course not. Because there are still millions of people that enjoy a football match or beer without getting crazy. Should those pay for the deeds of a “few” idiots? If we would banish alcohol and football, we’d better start to forbid about anything. Violent news items, boxing, nudity, radio, controvert art that might hurt the feelings of some... There is a name for systems like that: dictatorship. Just censor and forbid everything. Problems solved.

The funny thing is that guys like Ernst have a quite strict religious background. Erm… That’s fine with me, but if one thing caused millions of deaths and intolerance throughout the past, it is religion. That man wants to protect our children for games and other impure things, but it’s perfectly fine if they read about unbelievers burning in hell in the Bible, Quran or another religious book. Well, Southpark perfectly covers a lot of these issues. And again, people that feel offended by that series often don’t get the clue at all.


Not solving the true problems:
We can blame games, internet or Marilyn Manson. But why do we never look at ourselves? Was everything perfectly fine if the Columbine boys didn’t get a Nintendo? Of course not, their problems lied much deeper. Think about their environment, the way they got raised, problems at school, and so on. If it wasn’t the game that triggered them, something else would have done it.

Violence and lusts are in our nature. Therefore we look for ways to ventilate. Get drunk, go to a bondage club, beat your wife behind the curtains, go kickboxing or fitnessing, watch Predator on TV, or control yourself by strictly following a religion/lifestyle. Most peoply can do this very well luckily, others have little problems… There is no way to prevent this, other than trying to raise our kids the best we can. They have to learn how to deal with problems, desires and temptations. Taming the beast is an ever existing challenge, with or without games.

Forbidding everything is a way of shoving all the garbage under a carpet. If you can’t see it, it doesn’t exist. But what happens if the crap starts peeling out anyway? Typically people that never faced a certain problem or temptation will react frantic. Simply because they can’t place it. Religious or dictatorship systems try to control this with punishment (hell, captivity), but I rather lead my way based on sense instead of fear.

Just another pic of that room, including a test monster. I which I had more rooms to show, but I only have one pair of hands! About the floor reflection, a cubemap nearby the camera is updated each cycle, allowing realtime reflections on any surface. The cubemap is blurred and downscaled a few times so objects can pick a blurry reflection as well. I still have to implement mirroring for 100% accurate reflections though (for water and such).

Well I can write another thousand pages on subjects like these, but I think you get the point. I’d like to add though that they simply don’t have the right to decide for me what's good or bad. Games are just another form of art and demonstrate one of Westerns great goods; freedom of expression. If our leaders start messing with that, we’ll get ourselves bombarded back into the dark ages.

So Ernst, mind your own business. I know there are bad parents that let their kids play violent games all day without ever doing a check up, but I’m not the one to blame for that. Rather than restricting everybody, you’d better concentrate on bad upbringing. I’m not forcing you to play violent games or to watch porn, so please don’t force me to stop neither. Thank you.

Balls of steel

Apologies for the missing post yesterday. As you may guess, I had to watch the football grand finale, ... which we lost again. That damn octopus was right again. Honestly, Espanol was dominating the match most of the time and probably deserves the title, but their goal in the very last minutes was "discussable". Any other sport is using camera's and R2D2’s to prevent refugee mistakes, but FIFA knows better. Oh well, no hard feelings. Although someone else in our street decided to start shouting and arguing with his girl on the middle of the street at 00:00. After all, our loss was her fault of course.

We have to do it with a bittersweet second place. Maybe another time, we were patient the last 32 as well. Ah, at least we had a good time, and that's what matters. Another enjoyment for this week is a second Follower on this blog. I'm glad people are still reading. It's quite a job to create a story and some visuals each week, and I'll have to admit it even brings some stress sometimes on Sunday. "Come to bed honey". "Just 5 minutes, I'm finishing my story!". And before you know it's past 00:00 again. No one judges or gives me deadlines, but this works like doing a diet. As soon as you start skipping or finding excuses, the whole thing will collapse sooner or later. "Must-write-some-thing". So, it really encourages to see people are reading or giving feedback.


Made a (retarded)Smiley model for testing purposes. First it is rendered with default lighting (Lambert, Blinn). On the right side a "red clay" material is applied, inspired on those cool ZBrush shaders. So how did we give the clayish look?

Ok, so what happened last week? If you had a careful look at the monster from the previous post, you might notice it's skin doesn't really look... skinny. Of course, there are plenty of ways to improve that. Draw a better texture, use another (detail)normalMap, tweak some parameters, and so on. But also the lighting plays an important role here. If you ever wrote a shader, you probably know the basic formula's to calculate Diffuse and Specular lighting the "Lambertian" and "Phong"/"Blinn" way:

   diffuseTerm = dotProduct( lightVector, surfaceNormal )
   halfAngle = normalize( lightVector + eyeVector )
   specularTerm = pow( dot(halfAngle, surfaceNormal ) , shininess )

If those are hieroglyphics for you, try to think of it like this. When light hits an object, it can get scattered into all directions due a rough surface (on microscopic levels), or it reflects straightly. Very diffusive materials are concrete, chalk or sand. Specular materials on the other hand could be metal, plastic or a mirror (perfect specular).

The basic lighting formula can be tweaked to some extent by giving it less or more specular, but it lacks on rendering more specific materials such as brushed metal, satin, clay or flesh. There are plenty of demo's that use alternative techniques such as BRDF, anisotropic lighting, Oren Nayar, and so on. However, smoothly implementing them into an engine isn't so easy. Especially not if you are doing Deferred Lighting like I do. Deferred what?

I won't go into detail, but Deferred Lighting basically renders the material properties of each pixel into a few target textures. Later on, the volumes of lights that affect your screen are placed on top and will use the pixel properties to do their lighting.

Four funky textures used for the Deferred Lighting process. Ow, if you wonder how I do transparent surfaces; good old forward rendering

When rendering the light volumes, you won't know what kind of material you are affecting unless the textures have information about it. The problem is that the amount of values is limited with Deferred Lighting (16 scalars in my case), so you can't pass big formula's or image data. On top, the lighting shader shouldn't be branching ( IF THEN else ) too much. In the ideal situation, you use one uniform lighting method that can solve all types of materials.


To fix these problems, I decided to use a Material-Lookup-Texture. Within this 2D texture, I can define up to 63 materials. Wood, chalk, gold, metal, clay, velvet, slime, poop, whatever. Each material has a section inside that (atlas) texture where it's Diffuse, Specular, RIM and Fresnel settings are stored. Possibly more parameters to come. The lookup coordinates will depend on the results of the basic lighting formula and the materialID that is stored in the Deferred textures. This
ID simply shifts the coordinates into one of the 64 regions.

   // To the basic stuff
   diffuseTerm = dotProduct( lightVector, surfaceNormal )
   halfAngle = normalize( lightVector + eyeVector )
   specularTerm = pow( dot(halfAngle, surfaceNormal ) , shininess )

   // Instead of using the results above, override them by peeking in our material texture
   Texcoords.x = diffuseTerm * scaleX + materialID_shiftX;
   Texcoords.y = specularTerm * scaleY + materialID_shiftY;

   // Go Gadget go
   materialData = tex2D( materialTex, texcoords );
   newDiffuse.rgb  = materialData.rgb;
   newSpecular.rgb = newDiffuse.rgb * materialData.a;
   // Same thing for RIM and Fresnel

This allows to use alternate lighting without expensive shader techniques or scarifying many material parameters in the Deferred Rendering pipeline. Since the lookup textures contain 2D data, it's also possible to make "weird" specular highlights or colorize the diffuse lighting. Think about satin or rainbow effects on glass/CD's.

Basic metal material applied here. Much specular lighting, almost no diffuse lighting. The glossy reflection is made with a blurred cubeMap and high frequency normal disturbance. More important, the ugly greenish bathroom setting (not finished yet) is inspired on "google images: Soviet bathroom". Always price if you type "Soviet" in front.

The only remaining problem is... How in st.Beards name do we define those lookup textures? I have about two options:
A.- Buy or make myself a, Jesus Christ, gonioreflectometer. These devices will measure reflecting light on a piece of surface from all kinds of angles.
B.- Draw the (gradient) textures myself, physically based on absolutely nothing.

So I chose the second approach. Since my paintwork are just wild guesses, I made a few tools that will compose the Atlas lookup texture and renders the results for a quick review though. All in all I'm not 100% happy yet, and the specific lighting results are pretty much washed out after combining them with plenty of other tricks (texturing, ambient light, reflections, SSAO, screen filters, and so on). but at least alternate lighting is integrated into the (Deferred Lighting) engine pretty smoothly now, for a rather low cost.

My floating "Segway" kinda like monster again, with a slightly new jacket. More diffuse, less shadow. Furthermore, the red edges are the results of RIM lighting, which appears when a light is shining right behind an object. It may be overdone here, but it somewhat simulates the skin translucency. In the meanwhile, I'm making the big brother of this character for the movie... Let's hope it looks somewhat more frightening.

Make it clay baby

Another busy week. Had to jump on the bike 05:30 already. What kind of times are that for an office guy like me? My programming shifts usually go from 20:00 to 01:30 (with some girlfriend and Family Guy breaks), so I had a few short nights this week. Oh well, I don’t really have a nine-to-five job anyway, and I don’t mind swapping my stinky programming hole for a factory or the fields.

Either way, what a fantastic week it was. Beautiful weather, machine worked like a charm, one of our good friends got auntie again from a healthy baby, and of course… after many years the Dutchies finally taught the Brazilians a football lesson! I thought we were doomed after the first 10 horrible minutes, but all damage was fixed in the second half. And so was our spirit. Everybody went home half drunk after work (we watched the game there at a big screen) so then you know it must be good.

Tractor orgy. My little baby in front, harvesting bulbs. Our machine in front digs the earth and shakes off the sand by transporting everything with lots of violence to the unloadbelt at the rear. The second driving castle washes away the remaining earth (containers of sand aren't worth much), the tractor behind supplies water, and the other tractor on the side brings the stuff to a factory… or ‘45 warzones where they had bulbs for dinner. So next time when you give flowers to your girl, tell her how much work it was to get them. Romantic isn't it?

Even more good news comes from the creative corner. I can model a box or refrigerator in Lightwave, but I never understood how the Pro’s would create high-poly models. Working with curvy organic shapes is hard enough already, and how the hell do you deal with wireframes made of millions of polygons? Beats me. No doubt it’s possible with Lightwave or similar packages, but you really have to master your tools. And probably be very, very patient.

I still have to create another character for the movie though, so I had to come up with something. When I was younger, I spent quite some time claying with friends and my little brother. In fact, my first game attempt (sort of Final Fight / Double Dragon) was made with clay. Backgrounds and character sprites were made of clay and then photographed frame-by-frame. Lots of work, but also lots of fun. Too bad I don't have screenshots or source of it anymore. Anyway, wy isn’t there a modeling program that works in a same fashion?

Then I remembered a name… Brush this, ZBrush that, blabla. So I decided to have a quick look at the trial version. Didn't expect too much though. I'm like an old grandpa when it comes to new technologies. Back in the old days everything was better. See? I've been searching this crazy kids UI for hours just to create a cube. Dammit, how unlogic. Can't read those tiny letters. Who the hell is "metaball"?! Get off my lawn! I want to play Bingo. Where's my hat? I'm tired! There is Germans everywhere these days.

Wait a minute… It actually works! Sure, the completely different interface required a few training video’s (plenty of them on Youtube). But after ~5 hours I was already creating this “thing”.

Creating man-boobs was never so easy.

Haha. But seriously, after ~4 years Lightwave, I wouldn't even know where to start when I had to create an organic shape like this. ZBrush works radically different from Lightwave, Maya or Max. Instead of wrestling with polygons in the famous 4 viewports, you will be “sculpting”.

First you throw a basic shape to start with on your “placemat”. Can be compared with using primitives such as cubes or cylinders, except that the basic shapes here come with lots of polygons already. You won’t be looking at vertices and wireframes though. The next step is to pick a sculpting tool. It works like drawing in Photoshop & friends, but additionally you will be bumping, cutting, moving, bubbling and shifting the actual 3D geometry. By the way, at the same time you can draw colors as well, meaning you can create your textures at the very same time. Ideally when creating veins and such. Just draw a slightly bluish line and you have a bumpy vein.

The testing model in the engine. From ~2 million polygons to ~2800. To keep those grandma veins and nipples, a normalMap was baked.

I expected a long learning curve, teaching yourself a few hundred shortcut keys, and searching frustrated between dozens of tools. But I was ‘claying” pretty convenient results after a few hours already. Believe me, that is quite unique. All of this happens on super high-poly models, without you even realizing it. The nice thing about that is that you can downscale it later to game-model proportions and extract a high quality normalMap by comparing the differences between the low- and high poly version.

Making a game-model wasn’t so easy though, I ran out of memory a few times on my 4-gig 32 bit machine. In the end I still need to transfer the whole thing to Lightwave. Here I would make a few Level Of Detail variants for distant rendering, plus collision volumes for the physics. Too bad Lightwave chokes on huge OBJ (> million poly) models, so I had to downscale it first inside ZBrush. Plus I needed a way to create UV-maps as well. ZBrush satisfies these needs as well with two powerful plugins, Decimation Master and UV Master. These tools will let you downscale the model easily, and UV Master unwraps the model for you. In case you are afraid the seams being misplaced; you define the importance of regions simply by drawing on the model.

Just chilling on my (furry) carpet. Next step is to use a somewhat better skin shader, as I don't like the lighting here. But I already have a battle plan for next week...

Well, you have to try it for yourself. But what’s maybe even more important, I had a good time while sculpting. I like Lightwave, but I usually I get impatient after seeing boring wireframes for too long. ZBrush works in a fun way, with its intuitive controls and “what you see is what you get” approach. I probably never will be a very good digital modeler, but I think I can create a character or two with this. Got to hurry though, that trial doesn’t last forever!