Tuesday, May 24, 2011

Ou est le swimming pool? #1

What a shame, the world didn't end yet. And of course that old fool from Family Radio didn't admit he was wrong either. "What can I say, I'm not a Genius." Yeah, God works in mysterious ways, so maybe you'd shut up next time? Making up stories to scare the hordes of sheep, tssk. For those who spend all of their money before 21 May: HAHAHA. Read Darwin next time.

Sorry, I got carried away. Still a little bit angry about this documentary from Louis Theroux I just on Belgian TV one week ago:
The most hated family in America
Very cute, but fundamental idiots like grandpa Harold or those intolerant assholes from the link above caused dozens of wars, crusades and other stupidity throughout history. Claiming to be God’s messengers, but doing nothing but threatening with Hell, Doom, Sinfloods… Let me say this:
- For the mouth speaks what the heart is full of (Matthew 12) -
And now go jerk off yourselves.

To the happy news then. First, we get some more help on the modeling part. Another bandito from Spain, Sergi.
He’ll be making models, maps, and eventually some of the textures that are required for them. Let's hope he can boost the development speed! The other good news; I just started adding some water-effects.

Do you want to go to the plage with me?

H20, the magical substance that is believed to be the source of life. And the source of eye-candy when it comes to visual scenes. Where there is water, there is life. No wonder scientists are curious about what they might find on Europa, one of Jupiters moon's that is covered with a thick shell of ice.

And where there is water, there are decent graphics programmers (ahum). No doubt that water is one of the eye-catchers when it comes to showing your game(engine). Now I'm not a real expert on this matter, as the last time I implemented it was years ago. So I might missed a few new gadgets. But since one of the readers here asked for some water-rendering tips not too long ago, I thought why not writing a little tutorial? For young and old:

1.- Basic water techniques (non-shaders, beginners stuff)
2.- (Basic) shader tricks
3.- Adding reflections and refractions

Users who already did some water and shaders might skip part one (and two). Although some of the tricks are quite universal. I won't mangle too much with waves and fluid dynamics though, as I simply never did that before. Anyway, have fun!

You could try to implement an aqua volume that reflects and refracts light exactly physically correct. Or you just fake it with some tricks, with a wink to mister Fresnel. As usual, games tend to pick the second approach. And although water in games like Crysis or Halflife2 looks pretty real by now, we can nicely see how the fake tricks evolved from a blue texture to sparkling clear water.

Doom and Duke Nukem simply painted an animated(2 or 3 frames!) texture on the floors. So much for water. But pretty soon things evolved to a transparent water surface quad, with a moving wave texture rolling on it. You can laugh about it, but we have to start somewhere. Plus some hardware can't do much better anyway (The Wii? Just joking). Well, how do we do this?

- Make a quad(or a grid of quads) -> the water surface
- Apply a seamless (mipmapped) texture that "rolls" over the surface.
You can do this by accumulating the UV coordinates (glTexcoord2f( x+time, y+time )
- Disable specular and diffuse lighting before rendering. Don’t apply light, or
an ambient color only.
- Make it half transparent (glColor4f( rgb, transparency)
- Let it blend. Don’t forget to disable these settings after rendering the water btw!
glEnable( GL_BLEND );
glEnable( GL_ALPHA_TEST );

Done. You can't get it much simpler than that. But at least most players will understand they are dealing with water here. How about some more effects dude? Sure.

Multi Texturing & Caustics
Apply the wave texture twice, using multi-texturing. Let the second texture scale and move in a different direction. Eventually use a different (wave) texture.

Hey… if you can draw a second(or third, or fourth) texture on that water-plane… Then it’s also possible to draw a “caustics” texture onto the walls/floors/ceilings nearby the water-surface right? Sure, you can use the very same wave texture, grayscale it, and render it on top with additive blending modus. Man, you can even render it on the objects / characters that wade through the water.

Underwater fog
You can cut your world in 2 halfs: everything below and above the water surface. Everything rendered below the surface can use a much thicker fog value. This effect will make deeper or distant underwater polygons fade out in a basic color.
Use dark blue for an ocean, or brown for a muddy canal. If you want to do it really good, you'll need to use "height-fog". This is not a standard OpenGL feature, and requires a shader though, but a very simple one :)

Or instead of using traditional fog, you can simply decide the fog value based on the height of a vertex(or pixel, if you can use shaders).

fog = min( 1.0f, (watersurface.y - pixel.y) * thicknessFactor );

This would be a crude pixel-shader formula. If you can't use pixel-shaders, you can still do it per vertex. For each vertex you render, use glColor3f( fogRGB );, where fogRGB is computed based on the vertex height. However, this won't work very well if you have relative large polygons though. If you wonder in which order you should render things now:
1.- Stuff underwater with fat fog.
2.- Stuff above water with normal fog
3.- (Transparent) waterplane

Here the Swimming Pool map anno 2011. I only had a few textures to decorate it, and no other contents. But here some first water effects nevertheless. The brownish look is simply (height) fog. The deeper the pixel, the darker. I didn't make caustics yet by the way...

Waves (& LOD)
A single flat quad can’t make 3D wave shapes of course. Split your quad up in multiple smaller quads ("subdivide"). When rendering the water surface, you'll calculate the height(Y) coordinate for each vertex based on a wave formula. Now this can be damn tricky! I have zero experience with wave formula's, but I suspect it's a mixture of lot's of sines & factors. So let's keep the code simple, you can go figure out that formula yourself :)

function wave(x,y:single) : single;
// Very simple wave function...
// _elapsedTime just increases with the deltatime every cycle
result := cos(x + _elapsedTime) + sin(y + _elapsedTime*4) * 0.2;
For y:=0 to 100 do begin
glBegin( GL_QUAD_STRIP ); // Render horizontal rows of 1x1 meter quads
// Render the first 2 vertices on the left side
vertex1 := vec3(0, wave(0,y+0) , y+0);
vertex2 := vec3(0, wave(0,y+1) , y+1);
glTexcoord2f( 0 * uvScale, (y+0)*uvScale ); glVertex3fv( @vertex1 );
glTexcoord2f( 0 * uvScale, (y+1)*uvScale ); glVertex3fv( @vertex2 );
for x:=0 to 5 do begin
vertex1 := AffineVectorMake(x, wave(x,y+0) ,y+0);
vertex2 := AffineVectorMake(x, wave(x,y+1) ,y+1);
glTexcoord2f( x*uvScale, (y+0)*uvScale ); glVertex3fv(@vertex1 );
glTexcoord2f( x*uvScale, (y+1)*uvScale ); glVertex3fv(@vertex2 );
// Eventually calculate the normals here as well,
end; // for x
end; // for y

LOD (Level of Detail)
Wait a minute. I end up with a few billion vertices if I want to make waves on a 10x10 km water surface! So, decrease the detail for distant quads. If you look over an ocean, do you see any waves at the horizon? Unless you have a monster tsunami, you won't. Simple, because it's way too far to see such details. So neither do we have to render them.

The trick is to make the grid less dense at distances. But how? Again, I never did LOD on heightfields or water-surfaces so I have to apologize. But I might have some ideas though. From the top of my head:

1- Make a 3x3 initial grid. Around the camera(player)
2- Evaluate all (9) quads you just made
3- If 1 or more points from a quad are inside X meters from the camera (just check the distance, simple enough), subdivide the quad:
* don't do any (wave/heightMap) height calculations yet!
* don't think about triangles or OpenGL yet, just manage cells and points

4- Repeat step 2 & 3, but with a smaller radius around the camera.
Do this until the radius comes nearby the camera. The more steps you take, the
more divisions (but also the more work of course).

You don't have a 3D mesh yet, just a bunch of quads/points. Now you have to do 3 or 4 things to make soup out of this:

- Triangulate the shape (=make triangles)
- Calculate uv coordinates (interpolate)
u = ((point.x - fieldOffset.x) / totalFieldWidth) * repeatUcount
u = (parentQuad.leftVertex.u + parentQuad.rightVertex.y) / 2
- Calculate height ( y = getHeight(xz) or getWave(xz) )
- Optional, calculate normals (based on your surrounding point coordinates)

Sounds not too difficult, except for the triangulating part. You can keep track in each cell(quad) while dividing. Normally, a cell has 4 points on the corners. When subdividing, a new point will occur in the center, and 4 new cells will be generated. More tricky are cells that have extra points from a divided neighbor cell. See the picture for possible triangulating options.

LOD is a useful technique for large surfaces such as water with waves or heightMaps (terrains). The implementation above is just my quick thought. You might find better ways on the web.

Specular light
So far we didn't use any lights except an overall ambient color. Which might be a little bit odd of course. If you shine a flashlight on the bathtub, you won't see a nice circle spot caused by diffuse light. But what you do see is reflected light (=specular light). Well, sort off. Don't shoot me if that is not exactly correct, I slept during physics lessons.

Anyway, as water often comes with a sun or moon in outdoor scenes, you can activate specular light from that specific source. Be careful, you'll need sufficient triangles to do this (or use per-pixel lighting(and thus shaders)). That means you have to sub-divide your water quad. See LOD above. If your triangles are too big, the specular light effect will be chunky as well.

If you are doing real 3D waves, you will need to calculate correct vertex-normals as well, or your specular light suck.

Here you can clearly see the effect of specular light. Per-pixel specular with simplified Fresnel that is though.

I’ll keep it short. Don’t forget water splashes, underwater bubble sprites or drawing (additive transparent) ripple sprites on top of the surface whenever something falls in the water. There are advanced tricks for this as well, but with some simple sprites and particles you can come quite far already. Don’t underestimate the impact of these effects! You can have a horny Lagune-shader, but the illusion of water will still break in thousand shards when a fat man bombs himself into the water without making a splash.

So, enough for this long post. To be continued. Take two or three weeks to read it, cause I’m going to Poland this Sunday for a little vacation… If that fucking Volcano in Iceland let us go, that is…

Friday, May 13, 2011

Pina Collada

-Sorry if you see this post twice... Blogger suddenly removed it, so let's post it again-

Programmers are stubborn. nerdy, pimply, annoying persons. Ok, that might be too much of a stereotype, but stubborn we certainly are. No miracle, because when working with computers there are 7 million different ways leading to Rome. There are no single, simple, comprehensive solutions in computer-land. If someone asks how to implement a small administrative database for the local kebab store, You can’t just point to a single book which has the universal explanation for such a task.

For a start, what hardware/platform to chose? A MacBook? Unix station, PLC, Microcontroller, Industrial PC, Gameboy Advance? Next step, which programming language do we use? .NET, Java or are we going to print good old IBM Punch cards? Those who read some technical forums, know these discussions are guaranteed for some hot nerd-fights. Hail Linux! Windows 4Eva! Pic16F microchip for real yo. The funny thing is, after choosing your hardware and development platform, you still have nothing; the program has yet to be made. So, unless you need some very specific hardware, stop arguing and just pick the tools you are comfortable with, because it gives the best chance of actually succeeding.

Then when it comes to programming itself, again we have an infinite amount of strategies. OOP programming? Functional programming? Design patterns, or something very different? Do you use existing frameworks, or do you make your own components? A* pathfinding or Dijkstra pathfinding? OpenGL versus DirectX. XML files or binary files. Sigh.

Off topic. The Radar Station test-map in progress. This map will be a playground for the Object Editor. Plenty of stairs to torture ragdolls. This is a not an Engine render by the way.

Honestly I never really readed books about programming (except some Delphi beginner books a LONG time ago), and developed my own "style" throughout the years. A mixture of Karate, Jujitsu, and Turkish oil wrestling. And I guess that's how most programmers do it, more or less. Following advises from the books or not, at some point we'll have to decide whether to go left or right. And after X years you have a portfolio filled with a couple of languages, hardware, platforms and techniques you'd prefer. Which hopefully gives you a nose for things.

And that's how we become stubborn. Because with so many other techniques out there, we have to defend our chosen methods continuously. If some Cobol fanboy touches Delphi, he touches me. C'mon, you and me, after schooltime! Nah, but if you can't convince yourself walking the right path, you'll be lost in no time. It's not there are just one or two other alternatives you can try.... There are one or two THOUSAND alternatives to try, and each takes days, weeks or even years to master. It's just too much. You can ask around on the forums for some guidance (C++ or Java, OpenGL or DirectX?), but in the end you'll have to throw up a coin, or follow your heart. And don't forget that these forum guys, also the experts, are biased just as well. John Carmack’s advise will probably be very different from Tim Sweeney.

And of course, you'll be making wrong decisions now and then. Linux sucks after all ( :) ), Windows can't run 24/7 without a crash, OpenGL may become obsolete, DX doesn't run on your target platform, C++ was too difficult, Java too slow, Delphi not 64 bit. Whatever. But don't worry, making faults is a crucial part of the learning process. You can’t become a boxer without ever getting punched on the nose. Nevertheless... it hurts to discard a technique you spend many, many hours or even months on. Which is why programmers are even more stubborn than the average Joe. Don't admit your faults and fight till the very last man. Americans in Bagdad? Where? Russians in Berlin? We'll show them. Now you know why programmers have entrenched habits, and are deaf to criticism sometimes :)

You may have read the title, "Collada", and now you think "What the hell has a tropical drink / file format to do with the personal problems of a programmer?". Well, as said, Collada is a file format. For 3D related stuff to be more precise. These files can contain geometry data, materials, physical properties of a mesh, scene setup and animations I believe. Right, so we have 3DS, OBJ, LWO, MS3D, X and a few billion other 3D file formats. What format to chose?! Arh, another decision to make. And yes, it does matter what you pick. Some formats are more mature than others. And more important, you'll be telling which tools to use. If you make an *.MS3D (Milkshape) file importer, your modelers will need Milkshape or at least a (good working) exporter plug-in in their software. Unfortunately, plug-ins often include errors (outdated), and tossing around converted models from package to
package requires extra (annoying) time and is prone to mistakes.

A (love) letter to Julio who made this model. The trick is to put as much as info as possible in a model. Engine22 objects aren't just meshes with a texture. It also contains physical info (collision shapes & hitzones), Level of Detail variants so we can use a lower-resolution mesh for distant rendering, attachments such as lamps or particle generators. And eventually sub-objects that are connected with physical joints such as a door hinge.

This is why Collada was made; an attempt to create an universal, simple, all-round file format that will be supported by all 3D packages. Now I'm not sure if they actually accomplished that goal, but at least Blender and 3DS Max support this format, and newer version of Lightwave as well I believe. Yet, I made a Lightwave (LWO) importer ~6 years ago. A logical decision, since I used Lightwave. But from all formats, the LWO is one tough son of a bitch. And at this point, I'm forcing my fellow helpers (who use Blender and Max) to either find a LWO exporter, or we mess around with the OBJ format. Max to OBJ, to LWO, to Tower22. Bleh.

Wouldn't it be smarter to write a Collada importer then? Maybe, but this is where my stubborn, defensive programmer instinct comes around the corner again:
..A: I spend many f#cking hours in thet LWO importer, it finally works now :(
..B: Collada, pfff, don't let me laugh.

For one thing, I just don't like the name "Collada". It sounds like a tropical alcohol free drink or a kids icecream you buy on vacation. But what struck me more was the XML format. Yes, Collada files are just XML files. Which mean you can actually open the file as a text, or view it as a tree structure in any internet browser. I’m more a binary-format man. Never followed the Extensible Markup Language(XML) hype. Text formats… pfff. 50% of those files is overhead you don't need. Instead of just loading array's of data, you'll be "parsing" a whole pile of shit before you finally found something useful. It takes time, extra storage space, extra code to “understand” the file… No sir, I'm no fan of XML. And therefore neither of that Icecream format.

Fair enough? Maybe not. Just because I don't like it, doesn't mean it's rubbish. But it's just the way how my programming career got raised, people are afraid for changes, forgive me. Fact is that XML is used in many applications. Fact is also that Pina Collada is used in several 3D packages, at least more than my beloved Lightwave LWO format. Live with it. So, after many years, I finally decided to take a look at it... And with the Delphi TLibXMLparser it took me 6 whole hours to make a (basic) Collada importer for the Object Editor. The LWO format took me 6 light(wave)years...

Ok, ok, I've done this so many times that I can write file importers while sitting on the toilet, reading a Chuck Norris article. But I'll confess: I was wrong. I can hardly get it out my hands on the keyboard, but here you go: Boys and girls, if you need to pick a fileformat for an environment with a variety of 3D software packages, Collada is a good choice, and programming your own importer isn't so hard, certainly not if you can find a nice XML reader. Happy now?

There will be water and ice here. Gives a nice chance to play with water shaders and low-friction. You know what... I might just make a tech-movie once this map is up and running...

Monday, May 2, 2011

Smoke them out

Bin Laden dead? C'mon, put your AK-47’s in the air, scream it out on the streets, and do a victory dance. Ok enough silliness, let's just return to work. Osama dead eeyh... Now what? Asides from Gadaffi, and Kim Yong Pangang Il Turbo, there are no charismatic badguys at the moment. And... is the dead of Osama a symbolic thing, or will it really change the world? Unlike Hitler who was the "end-boss" of the WOII game, Al-Qaeda and the likes are usually independent cells. Osama was/is more like an inspirator, a motivator, a pop idol. Dead or alive.

I guess panic for terrorism is still there, and the real hardcore terrorist probably knows how to hate & how to detonate a bomb, even without uncle Bins. Anyway, if this Osama person really did what we he did (I know, there a lot of theories about him), he deserves to be dead. Not just for 9/11 and putting the Western and Islamic world into a dirty cagefight. Also for terrorizing Afghan people by using religion as an excuse for the 8th century barbaric Taliban ideology. Downgrading women to R2D2 service robots and forbidding kiting for kids. So... have a nice time in heaven with 72 lovely virgin piglets.

Or... maybe he got his paycheck, shaved his beard, shaked hands with the Navy Seals, and drove off in his Ferrari with a smile after ~25 years of outstanding acting work. It would fit in the complot theories. Who shall say. The story that his body has been dumped in the sea right after he was killed is odd, the least to say. Out of respect for Islamic rules that state Muslims have to be buried within 24 hours. Could be me, but I never saw soldiers cleaning up body pieces of Taliban that have been hit with a .50 in the desert. The meanest of them all, the uber-bombastic terror maestro, would get a fancy sea burial (is that allowed for Muslims anyway?). Out of respect… yeah right. With all those complot theories about 9/11 being an inside job, you would expect the Americans to show Bin Laden’s last minutes in full HD to prove the contrary. Or even better, catch him alive and “bring him to justice” like they did with Saddam. I hate complot theories, but this story smells a bit... like an old wet beard.

Sorry, I just don't trust politicians that much. Well, hopefully things will get explained soon. I think the world owes that, after 10 years of war and getting my underpants frisked at the airport. But assuming they really got him, it has to be said that the Americans kept their word. You can say all kinds of things about them, but not that they don't have the balls to accomplish big things. Other than we Dutchmen(& European in general) who prefer to criticize from the sidelines (like I just did now). Hey, guys like Osama don’t get smoked out just by talking you know. Talking about mysterious deaths, where the hell is Hitlers body?


Here, an old pic rendered by the great grandpa of our current engine, at times Saddam Hussain was still alive! This engine used Lightmaps btw, and a very special "pee shader" for that orange tube. Hey, got to show something.

Ok, how's Tower22 going? No new pics lately huh? True. But cheer up, plenty of things are being done. It's just that I'm not busy with any graphical programming tasks at the moment. In fact, the game doesn't even start at the moment as I'm doing all kinds of upgrades. And one update on X initiates two other updates on YZ. And before you know it, the whole source-code is turned inside out for the 600th time. I know, leaving your game “un-executable” for too long can be a critical point in the design traject. It's like stop cooling the Fukushima core for too long. But the planning is to have it back online within a month. With:

- FMOD Sound DLL
- Input DLL
- Newton2 Physics DLL
- A new Entity system (the stuff I wrote about in the "Puppeteer" posts)
- Several new features in the engine like trigger volumes or moveable platforms like elevators.
- A new Object Editor

Initially I was only implementing those custom DLL's to control the behavior of objects. But while doing so, you walk into all kinds of outdated stuff that had to be upgraded sooner or later anyway. Physics too old. Object Editor messy. Animation system sucks. Blablabla. So before you know it, you have a whole dozen of tasks.

One of the more “catchy” upgrades at the moment might be the Object Editor. To clarify, an object in T22 is a 3D mesh that can be duplicated and placed everywhere. Like furniture, decorations, barrels or other junk. Objects usually make use of physics, can be destroyed sometimes, or can be picked up for usage. More advanced types of objects are the animated ones, or the "thinking" one (which simply make use of a more advanced DLL/script to control them).

These objects have to be made somehow, starting with a 3D mesh. All those raw Lightwave, Max, Blender or whatever program meshes & animations have to be imported into the game somehow. So far the Engine22 Editor already had tools for that, but a little bit outdated. So, since we are driving the touristic route already, a few more or less miles don't matter anyway. It would be a nice chance to update some of the object aspects, like synchronizing it's collision volumes with a ragdoll (or vice-versa), adding Inverse Kinematics, using those AI / behavior DLL's, ...uh … pretty much everything.

Another good reason to make some work of it, is because my fellow modelers still need a tool to see/test their own work. I can promise all kind of things, but seeing is believing. So far I could only send them screenshots of their productions after being imported into the game. Nice vacation pictures, but a real modeler wants to play around and tweak his work inside a real game environment of course. I could send them the Map Editor which has been used for everything so far, but it's too huge and buggy to share yet. Programming experience tells not to release your products to the public until a monkey can use it. Giving half-baked crap to your clients/users will give them a wrong impression and distrust of the product.

Then again, I can't let them wait forever. So, I'll focus my Super Sayan level 2 energy on releasing a stand-alone object editor first. But it makes use of the same engine modules, so it should come with two extremely nice features:
- Physics testing (throw around your object through a room)
- Full-shaded graphics. What you see is what you get

So it's not just an editor which allows to import some meshes, and tweak a script. It should also be able to play with your object in a test-map that actually uses the same graphics as the real game. Sure, it takes some energy to produce such a thing. But if I get happy modelers as a return, it will be worth it. Really, don't underestimate the effect of feedback. A modeler, mapper, sound engineer or whatever kind of member should be able to see, hear, play, read or experience his work back into "the product", otherwise the motivation to keep producing may drop to zero at some point. So before inviting all kinds of people on your project, make sure they can get the right tools to work with, and give them some options to see their work in action. All work and no play makes Jack a dull boy.

For the same reason I’m a little bit quite last months. I guess most people who visit here just want to see pictures. But sorry, all the programming hours went into upgrading lot's of boring code modules and editors. And playing "Tower Defense" plus staring at the screen :). Well hopefully I can place some nice pictures of a test-map being used for that object editor soon! Since it should have a pool to test buoyancy graphics, I might just as well program some water-shaders as well...

Here, another prehistoric render. That was one of my last water-shaders, time for a boost! Oh, people who live in Oudenbosch, you may recognize that pool :)