Back to school: 3D Vectors

Foreword
This (beginners) tutorial is written for Engine22 users, but also if you just want to know more about making games, engines, and 3D graphics. More tutorials will follow for sure, and can be found on Engine.Fuel22.net/Help/ (and at the time of writing, the link isn't up yet!). Have fun!



Remember Vectors from physics classes? Don't worry, neither do I. No but seriously, I'm a loser when it comes to math. Some parents say their kid is like a sponge. I'm more like bowling ball, without holes. If I don't apply it immediately, my brain does a toilet flush and the gained knowledge exits the body through farts or nose pickles.

“Apply it immediately”... what healthy 16 year old applies Pythagoras or Fresnel lens formula's for his daily problems? Only Newton's gravity law seems to apply sometimes, when falling drunk of your bike. E=MC2 in your face, stupid. So, there we are. In a classroom, thinking about the next part in Half life(1), staring at girls boobs, drawing idiotic doodles on a piece of paper, waiting until you can finally leave that sweaty moldy reeking place. Somewhere in the background, an old teacher with glasses is drawing "arrows" on the chalkboard. Whatever man.

But who would have thought that those arrows are called "Vectors", and that I would use them on a daily base, 17 years later? Just as realistic as using the leapfrog you learned at gymnastics.


As said, I'm just not that good at math so I won't be able to explain all the fine details about the math behind vectors. Then again, not being a professor, I can hopefully teach you a few things in human-language. While I'll put on my glasses and walk to the chalkboard, please stop chewing bubble-gum, stop doodling on your note block, and pay attention class. It's not that hard really, and if you want to make a game, you'd better stop watching your neighbour’s boobs, and focus on the chalkboard. Ahum:

Points

Earth is flat, and coordinates in space are made of 3 components: x,y and z. Don't believe me? Look at your finger. Move it to the left. Now it moved into a negative X direction; your finger's X-coordinate decreased. Move up. Your finger's Y coordinate increased. And how about that Z component then? Simple, just move your finger backwards, away from you. Z, not from Zorro but "depth", coordinate increased. From your perspective at least. In your bedroom door opening, your mom is looking at you now, and thinking what the hell you're doing. From her perspective, that finger moved into different directions, and she is about to call an ambulance.



So, a position, also called "point", can be defined as {x,y,z}. A 3-component struct in programming terms, like this:

// pseudo Engine22 notation

eVec3 = record

       x,y,z : eFloat;
end;

And a point can be in different "spaces". In general, we have "local space" (or "model space") and "world space". The finger example explained the coordinates, relative to your own position. Move it up, and Y increases. If you were the centre of the universe -as your mom always says-, your body would be at {x:0, y:0, z:0}. But you're not. For a Chinese on the other side of the world, your finger is going down instead of up.

That is... if the Earth was the centre of the universe. Which isn't either. In real-life, coordinate systems are always sort of local. It really depends what you take as a centre point, and also what direction is accounted for "forward", "up" and "left". If you hang upside-down like a bat the whole day, "up" might get a different definition.

Games are a bit easier. We just take a random point as the centre. Or well, random... On a GTA map, you may decide the bottom/left corner, at sea level, would be {0,0,0}. For Tower22, the centre of the ground floor is the centre of the "world". Game-world. When going up, Y goes higher. But there also games/3D programs that take Z as "up". It's arbitrary, and it doesn't really matter just as long your 3D models and programming math all accept the same rules.

In a game-context, "local" coordinates usually apply on/within a 3D model. So we make a large outdoor area. And we place assets in them. Cars, light-posts, garbage containers, trees, cats, et cetera. Each object gets a "world coordinate". However, earlier while creating these objects (say we were modelling a car), we didn't know if/where and how these objects are placed within the world. We just pretend the world is gone, and now the centre of the car itself becomes the "centre of the world". So, probably the handbrake would be somewhere near that centre. The front bumper, is placed forward from the centre, thus having a larger Z coordinate. The rear bumper, "behind" us, would get a negative Z coordinate.

Now when we place this car in our world, we can rotate it 180 degrees. In local-car-space, the handbrake is still the centre, and the bumper is still forward at a higher Z coordinate. But in world-coordinates, you'll get a whole different picture.

So, in practice, every object in your world gets an "absolute", world-coordinate. Physics,  movement, placement, and all that kind of stuff will be calculated with world-positions, world-vectors, and world-matrices (more about that later). But sometimes, you also want to do something in local-space. Or convert from local- to world space, or vice-versa. Example. Your NPC has a sniper rifle, and wants to blow off your head. That happens. Your "head" is a sub-part of your "player-body" object. The NPC knows your player world-coordinate. But next, the exact head-position depends on what the player is doing. If he is in "prone" pose, the head will be low at the ground. If he is licking his own balls, his head will be somewhere in the middle of the whole body-object. The body-object, which drives such animations, knows the local head position. In order to make a good shot, blowing of your head while you were walking on hands with your legs around your neck, the local head coordinate needs to be transformed to world space.

Jesus, that sounds hard. Yeah, it kinda is, and I won't explain the math now. But to do so, you'll need vector & matrix transformation calculations. Which are fortunately very common in any game engine, so you don’t have to reinvent the wheel. For now I'll just illustrate what can be done

with points. And I think... that I'm pretty much done. But to finish this story, let's throw some random (Engine22) code examples to get an idea:

                // Misc. Point code

                function makeYourPointPlease( const x,y,z : eFloat ) : eVec3;

                begin

                               result.x := x;

                               result.y := y;

                               result.z := z;

                end;

               

                procedure movePointToTheLeft( var pnt : eVec3; const distance : eFloat );

                begin

                               pnt.x := pnt.x - distance;

                end;

                              

                function distanceBetween2Points( const A, B : eVec3) : eFloat;

                var x,y,z : eFloat;

                begin

                          x      := A.x - B.x;

                          y      := A.y - B.y;

                          z      := A.z - B.z;

                         result := Sqrt(x*x+y*y+z*z);

                end;

Vectors

A point is just a... point, somewhere in a space. And you mainly need them for

- A.I. navigation

- Physics (movement, jumping, collision detection, ...)

- Tell video-card WHERE to draw

- Animations

Now to the Vector. In terms of programming, we can notate it the same. In fact, points and vectors are often the same struct or class in engines. In Engine22, a "vector" can be a vector(duh), but also a point or RGB colour. Anyhow, vectors can be visualized as arrows. Yes, that old man with glasses on the chalkboard wasn't fooling after all. A vector defines one or two things:

                - Direction: arrow point to the left, upwards, a bit backwards, ...

                - Force: The vector length. The longer the vector-arrow, the faster it moves into that direction

Note that "normalized" vectors do not define a Force or Strength, only a direction. A normalized

vector is made in such a way that the Force/Strength/Length is always 1. So a vector pointing to the left, would be noted as {x:-1; y:0 z:0}. An upwards vector would be {x:0 y:+1 z:0}. Polygon “Normals” (the direction its facing), is an example of a normalized vector.

The Matrix

Normalized vectors are used a lot for defining directions in drawing, physics, and rays in graphics(shaders). You see, objects usually don't only have a point, but also a certain direction. Looking from helicopter perspective, you could rotate a car 360 degrees. This angle can be encoded into a direction vector. Although you would need 2 more vectors for the full picture, as you could also roll and flip this car, in case it crashes or something. This is where Matrices are used for. A (4x4) matrix defines 5 things:

                turn / roll / pitch  rotation vectors

                position

                scale

Basically a matrix is constructed from multiple vectors. In Engine22, every asset has a matrix. When you place a new object into your world, you define its position, you can rotate it, and eventually scale (shrink / enlarge) it. In other words, you'll be adjusting its matrix while moving it around your mouse or keyboard.

This same matrix is fed into the physics system, so it knows where everything is, and also

shaders will get this matrix so they can position your object correctly on the screen (if visible

at all).

And back to Vectors

Matrix-math is quite a bit harder than vector-math, so let's spare that for another time, and continue this tutorial with some practical programming examples that show how these direction vectors can be used. Let's move bitch:

                procedure movePointUpwards( var targetPoint : eVec3; const unitsUp : eFloat );

                begin

                               targetPoint.y := targetPoint.y + unitsUp;

                               // Note it would move down if you give a negative number

                end;

                procedure movePoint( var targetPoint : eVec3; const forceVector : eVec3 );

                begin

                               // Assume the forceVector is NOT normalized,

                               // thus having a length/strength as well

                               targetPoint.x := targetPoint.x + forceVector.x;

                               targetPoint.y := targetPoint.x + forceVector.y;

                               targetPoint.z := targetPoint.x + forceVector.z;

                end;

               

                procedure movePoint( var targetPoint : eVec3; const direction : eVec3; const speed : eFloat );

                begin

                               // Direction vector is normalized here, multiply with speed

                               targetPoint.x := targetPoint.x + direction.x * speed;

                               targetPoint.y := targetPoint.x + direction.y * speed;

                               targetPoint.z := targetPoint.x + direction.z * speed;

                end;

Units in 3D space

So... if we move a point "20" to the left... then how far did it move exactly? My old physics teacher would outrage when writing a number without its corresponding unit. The mass of this block would be "10" mister. "10 what?! 10 ounce? 10 grams? 10 donkeys?!". "Kilograms mister". "Then say so". And of course, he was right. But still an asshole.

The guys at OpenGL or DirectX didn't listen very well to their teachers though; there is no

pre-defined unit for 3D coordinates. They basically say: "You figure out.". So, if we move

that point "20" to the left, we should decide ourselves if those are inches, meters,

nautical sea miles, or donkeys. In Engine22, "1" = "1 meter". But in another program it could

just as well be millimeters. So, it's important to pick 1 standard for your unit system, and eventually scale up/down imported models that come from a different 3D package. Lightwave also uses meters, but 3D Max centimetres if I'm not mistaking, so an imported model would be 100x bigger if you forget to downsize.

                procedure checkInput( );

                var movementSpeed : eFloat;

                begin

                               movementSpeed := 3.0;  // meters

                              

                               // Move our player with the arrows, over the X and Z axis

                               if ( keyboard.leftArrowPressed ) then

                                               movePoint( player.position,  vec3(-movementSpeed, 0,0 ) );

                               if ( keyboard.rightArrowPressed ) then

                                               movePoint( player.position,  vec3(+movementSpeed, 0,0 ) );

                               if ( keyboard.downArrowPressed ) then

                                               movePoint( player.position,  vec3(0,0, -movementSpeed ) );

                               if ( keyboard.upArrowPressed ) then

                                               movePoint( player.position,  vec3(0,0, +movementSpeed ) );

                end;

Stop. DeltaTime!

The procedure above has a little problem. Assuming that your game runs at 60 frames per second. And you'll be checking input every cycle. Thus "checkInput" would be called 60 times per second. Since we move "3.0 meters" every time, your player will go in warp-drive; holding the key for 1 second would move him 3.0 * 60 = 180 meters per second! That's a bit too much, don't you think.

Now we could simply reduce the "movementSpeed", to avoid super-human speeds. 0.03 for example would lead to 0.03 x 60 = 1.8 meters per second.

But this still stinks. What if you run your game on slower computer, that can only reach a lousy 20 FPS at times? 0.03 x 20 = 0.6 meters per second. In the worst case, your FPS fluctuates, because you're downloading internet porn at the same time, causing hitches. Or how about this? You'll buy a quantum computer that easily reaches 1000 FPS. Now suddenly your guy moves with 30 meters per second!

Ever tried old DOS games on an emulator? You may have noticed that some games seem to play in fast-forward. That's because they didn't guard their speeds. One trick is to introduce a maximum FPS. Another (better) trick is to calculate "DeltaTime". In Engine22, most "update()" functions will get a "DeltaSecs" argument with them; the elapsed time in seconds between the current and previous cycle. So if we're at a steady 60 FPS, DeltaSecs would be 1/60 = 0.0166667. Now check this:

                procedure checkInput( const deltaSecs : eFloat );

                var movementSpeed : eFloat;

                begin

                               movementSpeed := 3.0 * deltaSecs;     // meters PER SECOND

                              

                               // Move our player with the arrows, over the X and Z axis

                               if ( keyboard.leftArrowPressed ) then

                                               movePoint( player.position,  vec3(-movementSpeed, 0,0 ) );

                               if ( keyboard.rightArrowPressed ) then

                                               movePoint( player.position,  vec3(+movementSpeed, 0,0 ) );

                               if ( keyboard.downArrowPressed ) then

                                               movePoint( player.position,  vec3(0,0, -movementSpeed ) );

                               if ( keyboard.upArrowPressed ) then

                                               movePoint( player.position,  vec3(0,0, +movementSpeed ) );

                end;

Problem fixed. We multiply our original "3" with the elapsed time, so it gets a small number, 0.05 if the FPS was 60. And how much is 60 times 0.05? Exactly, 3. We moved 3 meters over a second.

Vectors in Engine22

You saw some of the basics. How to set a point, apply some movement, a bit about local versus world-space. There is a lot more you can do with vectors and matrices though. And therefore one of the basic elements each game-engine should have, is a Vector Library. And so does Engine22 of course. You already saw the "eVec3" type a few times above, which stands for "eVector3". That thing is being used all over the place, and has a bunch of handy help functions.

eVec3 is just a record with 3 floats. Note there is also an integer/byte/float32 variant, as well as a eVec4 type that has an extra W component. One of the reasons Engine22 won't work anymore in Delphi7, is because I made thankful use of record operators & functions, which wasn't possible in the old days. The means you can call functions and do math with vectors:

                var v1, v2, vResult : eVec3;

                begin

                               vResult := v1 + v2;           // Sums up (v1.x + v2.x, y+y, z+z)

                               vResult := v1 - v2;            // Subtracts (v1.x - v2.x, ...)

                               vResult := v1 * v2;           // Multiplies (not CROSS!)

                               vResult := vec3( 10, -4.2, 90);     // Constructor

                               vResult := v1.norm();    // Get normalized vector

                               string  := v1.toString();

                               vResult.fromString( "10 -4,2 90" );

                               x             := v1.dot( v2 );// dot product

                               distance:= v1.dist( v2 ); // Distance between v1 and v2

                               lookDir := v1.lookat( targetPosition );    // Gets direction towards target

                               length  := v1.len();          // Get vector length/strength

                               if ( v1 = v2 ) then vectorsAreEqual;

                               if ( v1 > v2 ) then v1IsStrongerThanV2; // Compare forces

                               ...

                end;

Another nice feature is that you can use the same vectors for RGB (or eVec4 for RGBA) colors. Instead of typing "vector.x", you can type "vector.r" as well. It does the exact same thing, but makes more sense in terms of writing. Vector colors are often used as input parameters for shaders and graphics. Note that 0 stands for black, and 1 for 255 (white) here, as we use a floating point notation instead of bytes.

And about 3D models, did you know that...

3D models (the stuff you make in 3D Max, Lightwave, Blender, Maya, ...) are basically just arrays of vectors? A polygon is made of (typically 3) "vertices". Where a vertex could have a position, a texture coordinate, a normal-direction, maybe a tangent, maybe a weight? So, writing it down:

                Vertex = record

                               position               : eVec3;

                               textureCoord    : eVec2;               // Only 2 coordinates (U and V)

                               normal                 : eVec3;

                               tangent                               : eVec3;

                               weight                 : eVec3;

                end;

               

                ModelMesh = class

                               vertexCount      : eInt;

                               vertices               : array of Vertex;

                end;

It's not exactly how the Engine22 VertexArray classes look like, but it's not far away from that either. When loading a 3D model, you'll be making arrays of vectors. When rendering the model, you'll be pushing those arrays to your videocard. Just keep that in mind.

It's not exactly how the Engine22 VertexArray classes look like, but it's not far away from that either. When loading a 3D model, you'll be making arrays of vectors. When rendering the model, you'll be pushing those arrays to your videocard. Just keep that in mind.

So much stuff you can do with them. The E22_Util.Base.Vector unit is one of the most common included units. And even if you're not a math genious, you'll get comfortable with them quickly.

To conclude, our deadly sniper headshot:

                procedure sniperHeadShot( targetEntity : eES_Entity );

                var         localHeadPos    : eVec3;

                               worldHeadPos : eVec3;

                               localGunPos      : eVec3;

                               worldGunPos    : eVec3;

                              

                        targetDirection : eVec3;

                        targetDistance  : eFloat;

                        bulletPos    : eVec3;

                        traveledDist              : eFloat;

                begin

                               // Get absolute head position

                               localHeadPos    := targetEntity.getLocalPoint( HEAD );

                               worldHeadPos  := targetEntity.getMatrix().localPointToAbsolute( localHeadPos );

                              

                               // Get our gun position

                               localGunPos      := sniper.getLocalPoint( GUN_NOZZLE );

                               worldGunPos    := sniper.getMatrix().localPointToAbsolute( localGunPos );

                              

                               // Get the (world) direction and distance between our gun and the head

                               worldGunPos.distAndDirectionTo( worldHeadPos, targetDirection, targetDistance );

                              

                               // Simulate bullet path

                               bulletPos            := worldGunPos; // Start here

                               traveledDist       := 0.0;                   // Traveled distance so far

                               while ( not collided ) and ( traveledDist < targetDistance ) do

                               begin   

                                               // Check if our path is clear

                                               collided := world.rayTrace( bulletPos, targetDirection, bulletSpeedMS * deltaSecs );

                                              

                                               // Move forward, with the speed of a bullet

                                               bulletPos := bulletPos + targetDirection * bulletSpeedMS * deltaSecs;

                                               traveledDist := traveledDist + bulletSpeedMS * deltaSecs;

                               end; // while

                              

                               if ( traveledDist >= targetDist ) then begin

                                               // Reached the target

                                               targetEntity.dieMotherFucker();

                               end;

                end;

               

Of course, that's bullocks code, but it does give an impression of how to work with vectors. The vector-related functions shown are in E22 at least. 

Build a Pyramid boys

While upgrading (or re-writing really) the engine, I thought it would be nice to share more technical details. Tool development, implemented graphics techniques, engine decisions, et cetera. Since the Engine22 code will be public, its probably a good idea to let you know what's going on right? So, here a post. Oh, and if you wonder where this code can be found, we got web-space: . But please allow me to finish some first tutorials first.

I found it difficult to make a start though. Grabbing a random recently added shader or something isn't difficult, but I always prefer to write a story. With a beginning, and end. With climaxes and plot-twists. With a joke, and drama. So where would we begin this story? The development goes so rapidly, techniques tend to change all the time, and especially the "beginning" -the foundation- of an engine is a wide massive block of code and design strategies. As with most larger software packages, there is no single point to start. An engine is made of many individual or interconnected modules. Some modules can be completely isolated, meaning that they don't require code from another module, and can be easily recycled in any project. For example, a math library. Other modules do depend on other modules; they are "build on-top". Like a pyramid.


A graphics module for example will likely depend on OpenGL or DirectX, and maybe some wrapper module in between. It also depends on your design. When making a "Sound Module" for example, we could keep it stand-alone and just limit its functionality to, well, playing sound! And maybe loading libraries, streaming stuff, doing 3D occlusions, reverb effects... But we could also decide to make it on top of a "World" or "Entity Module", so the Sound module is aware of the map geometry and actors that can emit sound. Choosing the latter usually makes the objects within such a Sound Module more logical, closer to your needs. But on the other hand your Sound module gets much harder to use in another project, as it relies on non-sound-related pieces. And this is exactly where engine (or other software) design goes wrong.

Programmers tend to let their modules(class/functions/structs) do too much. "Because its handy". A Physics-Collider object for example, is primarily made to "bounce". Check if there is a collision between A and B. But hey?! If that happens, we could let the same Physics-Collider trigger sounds and impact particles as well. And reduce the object health if the impact was hard enough. All the logical responses to a collision nicely coded at one place... But wait a minute. Now your physics module is doing physics, as well as sound, graphics, AND game-related events (reducing health, eventually killing the object if it fell head-first from a building)? It's like asking the vegetables-man to not only sell vegetables, but also cook and serve them. It's wrong I tell you.

As logical as it sounds, this brings confusion. If one object does a terrible lot, while another isn't, it gets hard to predict who's going to do what. Should we cook our own diner, or is the vegetables man going to it? And what if we don't want him to do that? In the example above, the physics-collider triggers particles, sounds and reduces health. But what if we don't have such a thing as health, and what if we want very different next-gen effects instead of stupid particles? Long story short, the Physics Module should shut up and keep itself to physics. Let some other, higher-in-the-chain module decide what to do. This brings me back to a pyramid design. Now how does E22 look like, more or less?

I'm a bad pyramid builder.

At the lowest levels, we usually find "Platform" related modules that we can't alter. Depending on our platform choice(s), this is what we get. Obviously the fundaments are different when choosing a Gameboy or PS4. If we want multi-platform support, we'd better use 3rd party modules or write our own wrappers that can switch between platform specific modules. Engine22 doesn't really utilize much Windows functions, and the chosen 3rd party libraries (OpenGL, FMOD or OpenAL, Newton, LUA) aren't bound to a specific OS either. Yet it would require some recompilation and IFDEFs at the deeper layers to make it really cross-platform. At this point, there are no plans for non-Windows platforms yet by the way.

The mid-layers of the pyramid form Engine22. Each brick represents a module, or a separate BPL package in Delphi terms. Usually the lower we get, the module either gets more abstract or specializes itself into a single, very specific task. Low level modules don't implement much game-specific rules and boundaries, making them easier to recycle in a different, eventually non-game project. Sound just plays sound. It doesn't know its role in the bigger whole. But as we climb higher, the modules are gearing more towards an actual game. A 3D Horror survival game like Tower22 I should mention, although "Horror Survival" could just as well be "Romantic Puzzle" or "Kiddy Adventure". Yet, the more higher level modules we use, the more restrictions. Engine22 wasn't designed for a Need for Speed racing game, or a Real Time Strategy Command & Conquer game. Not that it's impossible, but you probably need some add-ons to the physics, or in case of a RTS, a different "World" module as these top-down worlds are a different cookie than our roaming (mainly indoor) worlds.

The high levels are either Demo programs, Engine22 tools, such as our Map Editor, or of course your own program. Your code is in charge of which modules to use, what content to load, and what the "game-rules" are. Engine22 provides all kinds of mechanics for AI pathfinding, weapon damage or doing mini-puzzles. But in the end, you still decide, what/where/how and when. In coding terms, you'll be coding the green blocks in the pyramid, and your artist (or yourself) are creating resources such as 3D models, world-maps, audio libraries, textures, and so on. The stuff that makes a game.


Ok. Obviously the way how this pyramid is constructed, is pretty vital if your plan is to make an easy-going, robust engine. Wrong relations between modules will reduce reusability. Ambiguous module content will make the engine hard to comprehend and master for other users. Making it too abstract won't accelerate end-users in making their game much. Focussing it too much on a very specific goal will filter out potential users as they can't use it for their own purposes. Yin and Yang balance is key my friends.

The Engine22 design is pretty much done. Probably I miss a couple of modules later on the ride, but this pyramid picture isn't far away from reality. Any specific why's here? Nah, not really. It's based on accumulated knowledge (& failure) from previous engine attempts and using other libraries - not necessarily game libraries. If I learned one major thing about design, is that ease & logic sense double matters. If I have to study some class or function longer than a minute, it sucks. Bottom line. Of course, it’s impossible to make each and every function "simple". Especially in a large, complex, multi-discipline case such as a game-engine. But at least we should try to keep things as simple as possible. Keep the names, relations and purpose of your modules/types/functions simple. And it's often better to provide just one or two "working" ways to accomplish X, instead of drowning the end-user with a billion different ways. Last but not least, tools, examples, documents and somebody to ask your questions will make a difference.


The first bricks
So, what should I be writing first about this engine? You could just as well ask which brick to lay down first for that pyramid. There is no single point to start. But I tried to start with the "fully isolated" modules first. With that I mean modules that don't require another. See that "Util" module? That was my first one; Basic types, Vector types, Matrices, and various (string/math/system) functions. Simply, but really useful. And boring. If you want me to write about engine stuff, you probably prefer awesome stuff like Physically Based Rendering or monsters fighting with each other. But the sad truth is that we'll have to start with the boring basic blocks first. There won't be any elite troops rappelling down from a cliff as long as we don't have:


If just one of those components is missing, the scene won't work or looks like shit. Realizing that, making a game can be demotivating. Having to make three billion other things first before you can even do relative simple things. But people often assume that all of this background stuff is "just there". But really, no. It takes a mountain of "boring" code -stuff you can't show in a slick screenshot- first... Exactly a good reason to pick an existing engine that did (most) of that stuff for you already.

As an engine-user, you don't have to know what happens behind the curtains. Sure, every extra bit of knowledge makes you a better person, but since women can drive cars without even the slightest idea of the motor-vehicle concept, so should you be able to make a game or 3D project without having to understand every detail. So in that perspective, maybe I shouldn't bother you too much with the deeper layers of Engine22, and focus more on the pretty things that can be produce with it. Only problem... I'm not that far yet! At least not on a level we reached with the previous Tower22 engine, which was capable of showing (seemingly) finished scenery. And in case you're that kind of guy who knows what's happening under the hood of his car, you may actually do want to know the "boring bits".



Modules, modules... what's a module?!
All right then. I already wasted a few pages on the introduction, a bit too late to dive into a specific engine module. So instead, let's explain what an Engine22 module itself is then. When I started coding, I had to decide how these "modules" would be made, programmatically. You see, there are many ways to put it all together. Some call it "modules", some call it "libraries". Some call it "API", some a "Framework", others "SDK" or "Toolkit". And what they exactly are in terms of programming, can also differ. We could simply attach a bunch of code files, use DLL's (Dynamic Link Libraries), or use "Components" in the Delphi environment. Now what's it gonna be boy?

Probably you already got it, but let's start with my (not "THE") definition of a Module. Here it goes: A collection of functions or types that can be used to achieve a specific goal. Do physics, do 3D rendering, do math, do scripts, whatever. A module doesn't do anything by itself, it has to be utilized by some other (higher) module or application. We try to setup modules as isolated, thus with as little dependencies on others as possible. Modules are programmed as separate projects, thus not as a bunch of cooperating files in a larger project. Ok? Fine.


Now in general, there are 2 ways to include a module; Statically or Dynamically. Static linking basically means all the projects get merged into one executable. Dynamic linking will keep the modules separate, so your executable has to link with them when it starts (or later if you prefer, eventually optionally).

Probably I get my ass kicked by some of the stated pro's and con's, but it should sum it up more or less. To me, the biggest advantage of using DLL's, is that we can use them in other Non-Delphi programs as well. Java, C, .NET, your choice. You can't statically include Delphi code into a C++ project (although... maybe with Embarcadero C++ Builder...). Yet I chose for a Statically linked approach. Why? Mainly because "Eazy doez it", and because I would lose a lot of OOP/Delphi features otherwise. There are work-arounds, but basically you can't share a TSomeClass. If you look into the OpenGL or Newton API for example, you'll notice this is merely a bunch of (C compatible) functions. Nothing wrong with that, but for a game with lots of "real-life-like" entities, I found objects & classes more intuitive. Besides, having Engine22 working in a C++/Java/Whatever environment is not an absolute must. Plenty of other *good* engines for these languages out there. And last but not least, Engine22 has an API module that still allows scripts and programs written in another language to use the engine. Ultimately, wrappers can be made of course.

Anyhow, Statically linked modules it is. As for Delphi, each Module is made as a BPL package project. You include E22 in your program by simply adding the "/source/" folder of each required Module in your project "Search folders". And then include the units you'll need, usually starting with the "Core" unit that acts as a main entry point to all other modules and stuff. Well, don't worry, examples will follow.


Deploy
Technically, I'm not really utilizing the capabilities of a Delphi BPL package. Honestly, being new in Delphi XE as well, I wouldn't even know how to properly do that! (so Delphi guru's, help is appreciated here!!). I guess there are nicer ways to just activate a package, instead of having to manually include all /source/ folders. But that's a detail. Once it has been set-up, you're ready to play. Oh, and since we're statically linking everything, notice that A: the Source will be open, and B: there won't be any additional library (DLL) files you'll have to deploy with your executable.

Well... almost none. Because a few 3rd party modules are actually linked dynamically. I'm talking about OpenGL (3D graphics), Newton (physics), LUA (scripting), and FMOD (sound, optional). Note that FMOD isn't free. Or at least, you can download & use it for free, but you'll need a license when using your program for commercial purposes. Now I like FMOD, but I should be providing support for a free alternative -OpenAL (not to be confused with OpenGL). As for OpenGL for graphics, usually this is installed & updated automatically. But since E22 will be using modern techniques such as tessellation and bindless graphics, you'll be needing a relative modern video-card as well. One that supports OpenGL 4.3 at least.


When can we download Engine22 & start doing stuff? Well, we got webspace , and I'm writing beginner tutorials as we speak. It's just the engine as a whole is far from finished, so I don't want to upload a very buggy version to begin with. Then again it shouldn't take long anymore, just as long you don't expect a full working right from the start. If you do want a solid version, it's probably better to wait a bit longer :)

Certainly not back at the old Tower22 level, but at least there is progress. And quite rapid progress I should say. What you're seeing here? A shot of the Map Editor, Parallax corrected Cubemap reflections, a bit tessellation, HDR, SSDO (improved SSAO), Tiled Deferred Rendering, and some first bits on Image Based Lighting (IBL), using the Cook Torrance lighting BRDF and probes to sample reflections and irradiance. Now that's a mouthful. And Yes, this editor will become available too, but likely a bit later than the source code.

Play with Duplo

As promised, tutorials on making a simple game with Delphi & Engine22 are in the make. Plus I just bought some web-space. A download link to Engine22 or some examples would be pretty useful as well, don't you think?

But anyhow, it's not there yet, so let's write about something else in the meanwhile. Cats. Or horses maybe? Or let me explain one of the new tools in Engine22; "WorldEd" (World Editor). Of course, I didn't restart writing the engine for Tower22 if there weren't some good improvement plans. One of them had to do with the work-flow for making maps. Note that with a "map", I mean a piece of world. In our case those are mainly corridors, moldy Soviet apartments and hellish rooms. Or actually, we call them "Sectors" but that's just tech-slang dude.


Map-Editor
Making maps is nothing new in games. Take some 3D modeler or map-builder program provided with an engine, and do your thing. Doom and Duke Nukem did it, UnrealEngine does it, and so does Engine22. With one important difference though. Our Map Editor isn’t a full featured 3D modeler. You can’t make new polygons or primitives to form the walls, ceilings, stairs, pillars, mountains or whatsoever.

Instead, we use an ordinary 3D modeler (I used Lightwave, but Maya, Blender, Max, or any other program that can produce OBJ files will do). Why? Well, for three reasons:

1.- Making a *good* editor is a whole project on itself. I simply wouldn’t have time for that.
2.- And why would you, with so many *good*, mature, programs out there? Let the artist chose his or her favorite modeler instead.
3.- I want advanced, arbitrary maps if needed. Not just some box-shaped rooms made out of primitives.

Clearly, there are plenty of programs doing an awesome job in creating a map. So instead of trying to do the same, our Map Editor just imports files from these programs, and then does the finishing touch. We can move and weld vertices, adjust UV maps, put different materials. And moreover, insert entities such as lightsources, objects (furniture, junk, ..), decals or informative nodes for A.I. and such. Basically we make the geometry(mesh) in an external program, and accomplish it with the Map Editor. And that works pretty well.

BUT! There is always a but!. BUT what if you need to modify something later on? It’s not that we always exactly know what to make at forehand. Depending on the type of game, maps require tweaking. If you make a multiplayer map, you may need to increase or decrease coverage. If you make a platformer, you will most definitely fall down to death more than once before the distances and heights are properly set. If you make a spooky game with narrow corridors, the dimensions of those corridors may need to be corrected to avoid corridors getting too long, short, boring or whatever.

This is how Map Editors looked like 20 years ago. Besides the lack of a realtime 3D preview window in most editors, one big difference was that these first 3D games didn't support overlapping/multi-store geometry. So everything could be edited from a top-down view. Nowadays the geometry is much more complicated. Nevertheless, it was this (Duke Nukem 3D "Build") editor in particular that got me into "making games".


Yes, mapping takes some trial & error. A workflow that requires to import files, then re-import and start over again whenever something has to change, isn’t very well thought obviously. The new Map Editor fixes this partially by allowing re-imports without discarding all progress. The map geometry, or very specific chunks of the map can be reloaded while the rest of the map-contents stays intact. Plus it is also possible to export the map back to a workfile for your 3D modeler (read an OBJ file) again.

Also note that “maps” (or actually “sectors”) are only small parts, not an entire level or world. In the worst case a piece of corridor or a single room has to be re-done. Having to import & export between 2 programs (the Engine22 Map Editor & your 3D modeler) remains a weaker link in the chain, but the amount of swaps has been reduced, and in return the artist can keep using his favorite modeling software, which is also worth a penny or two.


From Concept to Map
We talked a bit about mapping and shit, but we often forget one thing in this whole misery: the preparation stage. Or as I like to call it, map-building on macro level. A good plan is half the work, and therefore also avoids a random “maybe try this” approach on the mapping. Not that it’s wrong to try and experiment, but seriously, make a plan first.

This is even more important when working as a team. To begin with, it’s pretty handy if your team knows what we’re making. Maybe the audio engineer doesn’t care too much about which flower wallpaper being used in room 55, but being able to show him an overview of the map and tell what happens when and where, does help getting a better understanding.

Moreover, it’s not uncommon to have one or more map-designers that do the work on paper mainly, but don’t actually model the maps in detail. Looking at our own situation, usually I planned the maps on a global level. Want a Radar building with 3 stores, theme is ice,, snow, concrete and old damaged equipment. Start upstairs, go downstairs, meet a monster, end of demo. Next, concept artists bring the idea alive with cool sketches, “mood” drawings, and research. Yeah, research. What exactly would you find in an old Soviet alike radar station? Small but important details like how the power sockets looked like, what kind of computers, or architecture styles need to be defined if you want to do a good job.


Note that concept artists typically don’t know/care about the exact map contents. They just do some research, use their fantasy in addition, and make drawings. Concept-drawings, which isn’t necessarily a beautiful drawing, but a pile of sketches so we can pick a few good ideas, and discard the rest. Based on that, a more final idea is made. We know the theme and have some eye-catchers, but now we’ll have to make a *playable* map. Absorb the word “playable”, because a cool-looking map isn’t necessarily a playable map. Which is why concept artists, that get paid to think out-of-the-box, usually don’t mangle with mappers and 3D artists that have to filter and flavour these ideas with game rules, constraints, and technical boundaries. The sizes and dimensions of the world need to match with a certain play-length. The world needs to provide either coverage for bullets, dark corners for monsters, or puzzles for brainiacs. And oh yeah, it needs to be doable with the available shaders, engine and computer powers.

For that, I like to lay in bed. With an old fashioned pen and paper, scrabbling how the main-rooms should be connected with tunnels, stairs, bridges or whatsoever. Then I fall asleep, head on paper, and hopefully I still like and remember the ideas next day. Ifso, I’ll copy the map blueprints to a digital painting program. MS Paint used to be awesome for that due its simplicity, but Windows 7 & 8 completely ruined it by trying to make it MS Paint “Pro”. But anyway, once these blueprints are more or less final, its finally time for a 3D modeler to make the maps in detail. Unfortunately due the lack of manpower, that “modeler” usually was me, meaning the quality wasn’t that high.

Once the maps were made & imported into the game via our Map Editor, we decorated the maps further with textures, decals, lights, objects, particles, and so on. And in some cases, an artist would have a final critical look, improve by drawing over a screenshot or providing some different textures or objects to finalize the map. Voila.

Sometimes I'll model a simplistic 3D map as well to visualize the ideas. Be careful not to put too much energy in it though. Concepts are to be ditched, to prevent tunnel-vision.


"World" Editor
These first steps are rarely discussed game-engine terms. Plenty of Map Editors, but as said, you already passed the conceptual phase then. Or maybe you didn’t and have to figure out something cool on the fly. Now some people have such a vivid fantasy that this “on-the-go” approach just works, but in general I wouldn’t recommend it. Making a (proper) 3D mesh takes energy you know. If you realize an idea doesn’t work for whatever reason, you may have wasted quite some time… with an unwanted side-effect of not-letting-it-go. People tend to stick to their plan once they spend some energy on it, even if they know it can’t be saved really.

That’s where the concept-phase is for. Just like with concept-art, you shouldn’t spend many hours and fall in love with your own creation too quickly. It’s better to make a quick sketch, or better, multiple sketches. Let it rest for a moment, let somebody else take a look and judge, then pick. If there are too many doubts, restart. As long as you didn’t spend too many hours on it, letting it go won’t be that difficult, and better ideas will come in return.


Well, that missing link between concept and actual mapping work inspired me into making “WorldEd”. For clarity, Map-Editor is for mapping on micro(detail) level, World-Editor for mapping on macro(global) level. In WorldEd, you’re not dealing with geometric details, textures, slopes or stairs. All we need to know is which rooms, where, and how they are connected with each other. Globally.

One nice thing is that Engine22 needs to know this as well. Mainly for these three reasons:

• Boundaries
Each map, -no, Sector- has one or more primitive shapes (cubes, cylinders, …) that form a boundary. Thanks to these boundaries, the engine can determine for any random point where it is in the world (very quickly).

• Portals
Rooms are connected via portals. Typically windows, doors, gaps, or just open space. These portals are used for global A.I. pathfinding, as well as culling. The engine can make a list of visible sectors for a given camera perspective.

• Load-Lists
Tower22 is a roaming game, meaning that you don’t have a traditional level1,2,3 structure. It’s just one huge world really. But you can’t load everything at once (even if you have multiple, but large levels) into the computer RAM and video memory.

Load lists define which other sectors should be present (in memory) when standing in one specific sector. It also defines at which level of detail. Very distant structures such as mountains or skyscrapers might be visible throughout the world, but at a very low level. Thus with an utterly simple geometric shape and very little or low resolution textures.


Previously we had to define this info in the Map Editor. Which isn’t too logical, as the 3D mapper maybe shouldn’t or doesn’t want to care about such “global” details. Let him or her focus on making a beautiful map instead. So I moved that part to another “WorldEd”. It enforces / encourages to make a low-detail map first.


So, the map-designer first makes a model by simply using cubes and other primitives. In WorldEd, we structurize (groups & sector names) and create boundaries, portals and loading-lists. Note that these boundaries are used later on again when the mapper imports the detailed geometry. Like a bakery cuttong tool, only the polygons that fall inside the boundaries are imported for the sector we are currently working on. This allows the artist to make a single 3D file made out of multiple sectors, without having to care about boundaries or connections. Another little plus compared to the old engine.

Finally, WorldEd is more than just playing with larger Duplo bricks before the artists do a finer job with smaller Lego bricks. Not implemented yet, but I plan the insertion of texts, lines and symbols. Not for the actual game, but for the game-design. This allows designers to show reference photographs or hints like “grab key at A, then go to B” in the World Editor. Making it easier and more visual to explain the battleplans to others. And moreover, map designing is fun. So if we can combine the necessary (making boundaries/portals/loadlists) with the joys of making a game…

Enough for this time. I’ll definitely write again about WorldEd when it’s a further state. Remember this tool is part of Engine22, so you can use it too when picking E22 .

Oh Behave!

A different post was planned, but I can't take screenshots since I'm on an old back-up station. The keyboard of my laptop suddenly stopped working partially, so I could only write half a story. Probably a cable rupture or something, though it wouldn't surprise me if the repair-assholes decide to swap hard-drives and reset everything anyway (as they did before at HP for no reason).

So anyway, let's fill the gap with a post that doesn't require spectacular screenshots then; Scripting.

One of the many features that has to be brought back alive again in the Engine22 rebuild, is Scripting. The ability to run your custom code without needing a programming environment or recompiling the project. The old engine used Python for that in a dark past, the new engine will use LUA. Why the swap? Hmm, no extravaganza arguments really. I just didn't like the Python syntax personally, people say LUA is a little bit faster due its simplistic basis. And maybe the most important reason, I don't have a recent Python-wrapper for Delphi XE. Most likely there is one out there, but nevertheless you can't pick just any library with Delphi, as there often isn’t an up-to-date header file due the limited userbase of Delphi. You can make your own wrappers of course, but... it's not that I don't have anything else to do.


Well, LUA then. What can you do with it, and how is it integrated in Engine22? Let's explain the difference between a script and regular code. Say we have an enemy soldier, hanging around, minding his own business, till we enter the scene. Upon sight he should start doing things. Erh, soldier things, like drawing a gun and throwing grenades. But this particular soldier shall run towards an alarm-button first. First of all, note that this type of behavior isn't accepted in an Engine. Not that we don't tolerate grenade-throwing soldiers, but engines shouldn't be involved with such game-specific things in the first place. Engine22 doesn't know if your game includes smart soldiers, stupid zombies, or peace-loving Gummi bears. The engine doesn't have the grenade-throwing or run-to-alarm thing hardcoded. It only provides various functions to achieve this. Like finding the path to that button, testing if guy-A can see guy-B, or enabling a grenade-throw animation. Engines are abstract in this; a higher layer should decide what to do, when to do, and how to do.



This higher layer is *your* code, pasted on top of the engine. Basically the engine does the deeper "standard" stuff, and triggers events. Your code tells our soldier to keep an eye open for our other player character(s). On detection, the engine throws an "onSight" event. This event provides some technical arguments, such as who can be seen and how far away. But what will happen next completely relies on your custom code inside that event. Ignore the player? Start shooting? Say hello? Run and hide? You tell.


onShitHappens( x )
Each "entity" (could be a monster, could be a lightsource, could be a peeled banana) can carry a "Behavior" object. Often simple entities like junk or furniture share the same standard Behavior. This Behavior is a collection of events that can be called by the engine. Whether you use them or not is up to you. To give an idea what kind of events- onSpawn, onKilled, onUpdate, onSight, onCommand, onGetInterfaceOptions, onCollision, onPhysicsUpdate, and so on. When not filling in any behavior, your entity will fall back to standard engine code. Which is fast, but doesn't do much more than basic physics. For example, if your can-of-paint falls on the ground, it won't make clunky sounds, nor leave a paint decal. To do so, you should override its "onCollision" event. This event tells you the velocity, the angles, what kind of materials made contact (metal versus skull), which hitzone (knee, head, balls, rear-wheel), et cetera. Based on that data, do whatever is necessary.

Now you may think, why doesn't the engine just handle this by default, for the sake of simplicity? Making a hit-sound, putting a decal and bouncing back is quite obvious right? Well yeah, but what IF you want something more spicy? Absurd bouncing physics, increasing the score when hitting X, breaking the object in smaller bits... maybe the object shouldn't collide at all if it's a ghost. See, the engine can't predict your wicked plans, which is exactly why you should be able to override things with your own code.


This can be done in three ways; via Compiled Code, Scripts, or "Engine22 Triggers". Compiled Code means you override the Behavior class with your own class in Delphi, code whatever needs to be coded, and compile your project. Note that it will also be possible to let a Behavior point to another DLL as well, allowing you to write a C++ DLL. Either way, this means the behavior is hard-coded; each change requires a re-compilation of either your ProjectCode or DLL.

Scripts are a bit the opposite, as they can be parsed and executed at runtime. Basically you can type a piece of custom code anytime, and let it execute by the Lua Interpreter. If you don't like it, change the code and try again. No need to re-compile and restart. In general Lua is also simpler, which makes it more interesting for non-programmers. The syntax is more relaxed, and since the Engine guides you into filling small logical events ("onHitByBullet", "onPathTargetReached") you don't have to understand the overwhelming behind-the-scenes part.

Only downside is that scripts run slower, as the code needs to be compiled first. Compiled Code is (much) more efficient and powerful than Scripts. But again, they require a programming environment (Delphi, Visual Studio, ...) that not every (artist) user owns or understands. Moreover, it can be tedious to hard-code each and every event. A lot of events in a game will only happen once. Hitting "lightSwitch #634" that toggles "lightSource #1000" and "lightSource #1001", covers a piece of code that is only used by this single instance. If we had to make an overrided Behavior class for each switch or game-map-specific event, we'll end up with a big pile of sh*t, and lose lots of time while prototyping. Scripts are quick & flexible, but require some attention performance wise. Executing a single event is no biggie, but doing calling lots of scripts continously ("onUpdate" event) for many entities may eventually have its impact.

"Engine22 Triggers" is a set of standard conditions and events integrated in the engine. A trigger holds a list of conditions, such as "player stands inside X?", "timer > 60 seconds" or "Z is dead?". If all conditions are true, the trigger will execute its events. Events such as "play sound", "show/hide object", "set animation" or "execute script", where the latter allows you do run some custom Lua code. Engine Triggers are cool because they are efficient, and don't require any programming. But their usage is limited. Very advanced conditions or events may still require script. As said, its a set of common conditions & events.


Preview of the code editor, that allows to (de)activate specific events, and a API browsing tool to help finding the right functions.


API (Application Programming Interface)
One more thing to explain, is the E22 API. Obviously scripts shouldn't be doing too much, regarding performance. Engines are there to help you out, doing the hard stuff. Therefore both scripts, Delphi code, or DLL's in another language can make use of the E22 API, which is basically just a large list of various functions. Functions to move around entities, math & pathfinding functions, playings sounds, doing animations, spawning new entities, killing entities, et cetera.

Doing scripts is the fun part, as you actually define what happens in your game; game-code. All in all you should be able to customize quite a lot without really needing Delphi or another programming environment, which is a good thing. Now this post mainly showed the relation between “entities” (objects in the game world) and scripts, but also notice that scripts can be attached to overall game-rules or UI (“onButtonX-click”).

When are we going to see Engine22 btw? Well, the good news is that I finally bought some webspace so others can download. But before doing so, I want to finish some demo programs first, and moreover, test if another person –a Delphi beginner- can make a project & run things. Of course Engine22 is in a very, VERY, early stage. So releasing to a wider public would be a bit stupid I’d say. Likely I’ll be asking here for some more guinea pigs first.