I found it difficult to make a start though. Grabbing a random recently added shader or something isn't difficult, but I always prefer to write a story. With a beginning, and end. With climaxes and plot-twists. With a joke, and drama. So where would we begin this story? The development goes so rapidly, techniques tend to change all the time, and especially the "beginning" -the foundation- of an engine is a wide massive block of code and design strategies. As with most larger software packages, there is no single point to start. An engine is made of many individual or interconnected modules. Some modules can be completely isolated, meaning that they don't require code from another module, and can be easily recycled in any project. For example, a math library. Other modules do depend on other modules; they are "build on-top". Like a pyramid.
A graphics module for example will likely depend on OpenGL or DirectX, and maybe some wrapper module in between. It also depends on your design. When making a "Sound Module" for example, we could keep it stand-alone and just limit its functionality to, well, playing sound! And maybe loading libraries, streaming stuff, doing 3D occlusions, reverb effects... But we could also decide to make it on top of a "World" or "Entity Module", so the Sound module is aware of the map geometry and actors that can emit sound. Choosing the latter usually makes the objects within such a Sound Module more logical, closer to your needs. But on the other hand your Sound module gets much harder to use in another project, as it relies on non-sound-related pieces. And this is exactly where engine (or other software) design goes wrong.
Programmers tend to let their modules(class/functions/structs) do too much. "Because its handy". A Physics-Collider object for example, is primarily made to "bounce". Check if there is a collision between A and B. But hey?! If that happens, we could let the same Physics-Collider trigger sounds and impact particles as well. And reduce the object health if the impact was hard enough. All the logical responses to a collision nicely coded at one place... But wait a minute. Now your physics module is doing physics, as well as sound, graphics, AND game-related events (reducing health, eventually killing the object if it fell head-first from a building)? It's like asking the vegetables-man to not only sell vegetables, but also cook and serve them. It's wrong I tell you.
As logical as it sounds, this brings confusion. If one object does a terrible lot, while another isn't, it gets hard to predict who's going to do what. Should we cook our own diner, or is the vegetables man going to it? And what if we don't want him to do that? In the example above, the physics-collider triggers particles, sounds and reduces health. But what if we don't have such a thing as health, and what if we want very different next-gen effects instead of stupid particles? Long story short, the Physics Module should shut up and keep itself to physics. Let some other, higher-in-the-chain module decide what to do. This brings me back to a pyramid design. Now how does E22 look like, more or less?
I'm a bad pyramid builder.
At the lowest levels, we usually find "Platform" related modules that we can't alter. Depending on our platform choice(s), this is what we get. Obviously the fundaments are different when choosing a Gameboy or PS4. If we want multi-platform support, we'd better use 3rd party modules or write our own wrappers that can switch between platform specific modules. Engine22 doesn't really utilize much Windows functions, and the chosen 3rd party libraries (OpenGL, FMOD or OpenAL, Newton, LUA) aren't bound to a specific OS either. Yet it would require some recompilation and IFDEFs at the deeper layers to make it really cross-platform. At this point, there are no plans for non-Windows platforms yet by the way.
The mid-layers of the pyramid form Engine22. Each brick represents a module, or a separate BPL package in Delphi terms. Usually the lower we get, the module either gets more abstract or specializes itself into a single, very specific task. Low level modules don't implement much game-specific rules and boundaries, making them easier to recycle in a different, eventually non-game project. Sound just plays sound. It doesn't know its role in the bigger whole. But as we climb higher, the modules are gearing more towards an actual game. A 3D Horror survival game like Tower22 I should mention, although "Horror Survival" could just as well be "Romantic Puzzle" or "Kiddy Adventure". Yet, the more higher level modules we use, the more restrictions. Engine22 wasn't designed for a Need for Speed racing game, or a Real Time Strategy Command & Conquer game. Not that it's impossible, but you probably need some add-ons to the physics, or in case of a RTS, a different "World" module as these top-down worlds are a different cookie than our roaming (mainly indoor) worlds.
The high levels are either Demo programs, Engine22 tools, such as our Map Editor, or of course your own program. Your code is in charge of which modules to use, what content to load, and what the "game-rules" are. Engine22 provides all kinds of mechanics for AI pathfinding, weapon damage or doing mini-puzzles. But in the end, you still decide, what/where/how and when. In coding terms, you'll be coding the green blocks in the pyramid, and your artist (or yourself) are creating resources such as 3D models, world-maps, audio libraries, textures, and so on. The stuff that makes a game.
Ok. Obviously the way how this pyramid is constructed, is pretty vital if your plan is to make an easy-going, robust engine. Wrong relations between modules will reduce reusability. Ambiguous module content will make the engine hard to comprehend and master for other users. Making it too abstract won't accelerate end-users in making their game much. Focussing it too much on a very specific goal will filter out potential users as they can't use it for their own purposes. Yin and Yang balance is key my friends.
The Engine22 design is pretty much done. Probably I miss a couple of modules later on the ride, but this pyramid picture isn't far away from reality. Any specific why's here? Nah, not really. It's based on accumulated knowledge (& failure) from previous engine attempts and using other libraries - not necessarily game libraries. If I learned one major thing about design, is that ease & logic sense double matters. If I have to study some class or function longer than a minute, it sucks. Bottom line. Of course, it’s impossible to make each and every function "simple". Especially in a large, complex, multi-discipline case such as a game-engine. But at least we should try to keep things as simple as possible. Keep the names, relations and purpose of your modules/types/functions simple. And it's often better to provide just one or two "working" ways to accomplish X, instead of drowning the end-user with a billion different ways. Last but not least, tools, examples, documents and somebody to ask your questions will make a difference.
The first bricks
So, what should I be writing first about this engine? You could just as well ask which brick to lay down first for that pyramid. There is no single point to start. But I tried to start with the "fully isolated" modules first. With that I mean modules that don't require another. See that "Util" module? That was my first one; Basic types, Vector types, Matrices, and various (string/math/system) functions. Simply, but really useful. And boring. If you want me to write about engine stuff, you probably prefer awesome stuff like Physically Based Rendering or monsters fighting with each other. But the sad truth is that we'll have to start with the boring basic blocks first. There won't be any elite troops rappelling down from a cliff as long as we don't have:
If just one of those components is missing, the scene won't work or looks like shit. Realizing that, making a game can be demotivating. Having to make three billion other things first before you can even do relative simple things. But people often assume that all of this background stuff is "just there". But really, no. It takes a mountain of "boring" code -stuff you can't show in a slick screenshot- first... Exactly a good reason to pick an existing engine that did (most) of that stuff for you already.
As an engine-user, you don't have to know what happens behind the curtains. Sure, every extra bit of knowledge makes you a better person, but since women can drive cars without even the slightest idea of the motor-vehicle concept, so should you be able to make a game or 3D project without having to understand every detail. So in that perspective, maybe I shouldn't bother you too much with the deeper layers of Engine22, and focus more on the pretty things that can be produce with it. Only problem... I'm not that far yet! At least not on a level we reached with the previous Tower22 engine, which was capable of showing (seemingly) finished scenery. And in case you're that kind of guy who knows what's happening under the hood of his car, you may actually do want to know the "boring bits".
Modules, modules... what's a module?!
All right then. I already wasted a few pages on the introduction, a bit too late to dive into a specific engine module. So instead, let's explain what an Engine22 module itself is then. When I started coding, I had to decide how these "modules" would be made, programmatically. You see, there are many ways to put it all together. Some call it "modules", some call it "libraries". Some call it "API", some a "Framework", others "SDK" or "Toolkit". And what they exactly are in terms of programming, can also differ. We could simply attach a bunch of code files, use DLL's (Dynamic Link Libraries), or use "Components" in the Delphi environment. Now what's it gonna be boy?
Probably you already got it, but let's start with my (not "THE") definition of a Module. Here it goes: A collection of functions or types that can be used to achieve a specific goal. Do physics, do 3D rendering, do math, do scripts, whatever. A module doesn't do anything by itself, it has to be utilized by some other (higher) module or application. We try to setup modules as isolated, thus with as little dependencies on others as possible. Modules are programmed as separate projects, thus not as a bunch of cooperating files in a larger project. Ok? Fine.
Now in general, there are 2 ways to include a module; Statically or Dynamically. Static linking basically means all the projects get merged into one executable. Dynamic linking will keep the modules separate, so your executable has to link with them when it starts (or later if you prefer, eventually optionally).
Probably I get my ass kicked by some of the stated pro's and con's, but it should sum it up more or less. To me, the biggest advantage of using DLL's, is that we can use them in other Non-Delphi programs as well. Java, C, .NET, your choice. You can't statically include Delphi code into a C++ project (although... maybe with Embarcadero C++ Builder...). Yet I chose for a Statically linked approach. Why? Mainly because "Eazy doez it", and because I would lose a lot of OOP/Delphi features otherwise. There are work-arounds, but basically you can't share a TSomeClass. If you look into the OpenGL or Newton API for example, you'll notice this is merely a bunch of (C compatible) functions. Nothing wrong with that, but for a game with lots of "real-life-like" entities, I found objects & classes more intuitive. Besides, having Engine22 working in a C++/Java/Whatever environment is not an absolute must. Plenty of other *good* engines for these languages out there. And last but not least, Engine22 has an API module that still allows scripts and programs written in another language to use the engine. Ultimately, wrappers can be made of course.
Anyhow, Statically linked modules it is. As for Delphi, each Module is made as a BPL package project. You include E22 in your program by simply adding the "/source/" folder of each required Module in your project "Search folders". And then include the units you'll need, usually starting with the "Core" unit that acts as a main entry point to all other modules and stuff. Well, don't worry, examples will follow.
Deploy
Technically, I'm not really utilizing the capabilities of a Delphi BPL package. Honestly, being new in Delphi XE as well, I wouldn't even know how to properly do that! (so Delphi guru's, help is appreciated here!!). I guess there are nicer ways to just activate a package, instead of having to manually include all /source/ folders. But that's a detail. Once it has been set-up, you're ready to play. Oh, and since we're statically linking everything, notice that A: the Source will be open, and B: there won't be any additional library (DLL) files you'll have to deploy with your executable.
Well... almost none. Because a few 3rd party modules are actually linked dynamically. I'm talking about OpenGL (3D graphics), Newton (physics), LUA (scripting), and FMOD (sound, optional). Note that FMOD isn't free. Or at least, you can download & use it for free, but you'll need a license when using your program for commercial purposes. Now I like FMOD, but I should be providing support for a free alternative -OpenAL (not to be confused with OpenGL). As for OpenGL for graphics, usually this is installed & updated automatically. But since E22 will be using modern techniques such as tessellation and bindless graphics, you'll be needing a relative modern video-card as well. One that supports OpenGL 4.3 at least.
When can we download Engine22 & start doing stuff? Well, we got webspace , and I'm writing beginner tutorials as we speak. It's just the engine as a whole is far from finished, so I don't want to upload a very buggy version to begin with. Then again it shouldn't take long anymore, just as long you don't expect a full working right from the start. If you do want a solid version, it's probably better to wait a bit longer :)
Certainly not back at the old Tower22 level, but at least there is progress. And quite rapid progress I should say. What you're seeing here? A shot of the Map Editor, Parallax corrected Cubemap reflections, a bit tessellation, HDR, SSDO (improved SSAO), Tiled Deferred Rendering, and some first bits on Image Based Lighting (IBL), using the Cook Torrance lighting BRDF and probes to sample reflections and irradiance. Now that's a mouthful. And Yes, this editor will become available too, but likely a bit later than the source code.
