Start the engines

Enjoyed the movie from the last post? I didn't forward it to any websites (yet), so it didn't exactly attract a whopping audience, but I hope you had a couple of nice Youtube-minutes nevertheless ;) Keep in mind, an official T22 movie Is still in the make, and plans are to launch that one together with the Fuel22 website. About Engine22 then. As you may have red in previous posts, I'm putting a lot of programming hours into a rebuild of the engine. And not just a rebuild; a public version! For you, for me, for Delphi programmers, for non-programmers. Why?

* Because the old engine requires a rebuild anyway (64 bits, Physical Based Rendering, cleaning up old mess, newer Delphi version, stepping over to GL4.5 & GLSL, IBL / Tessellation, better design, ...)
* Because guys like you might be able to contribute by reporting bugs, request useful features, make demo's, or even contribute tools or core-code - I can't do it all alone anymore!
* Because it boosts my effort into making a *good* engine, including some tutorials and documentation (I’d better not make a fool of myself when showing off my code!)
* Because any kind of attention is good for the Tower22 project on the longer term
* Because Delphi deserves more attention as well. These are my 49 cents.

Right. I'm not at the point of giving you a download link yet. Mainly because I want to provide at least a somewhat solid basis, and because we don't have a proper website for these downloads yet. It will be integrated with the Fuel22 website, which is also still under development. But that doesn't mean nothing has been done yet! I'm certainly not the best programmer out there, but I sure can crap some bits at a high baud-rate once I get started. And also because many of the techniques have been done before in the "old" engine, a lot of time can be saved. In my experience, more than half of the development time is "wasted" on research, fixing bugs and maintenance. Doing a rebuild avoids some of those mountains.


Here, catch. First work on the "E22_GUI" module. It's for making user interfaces, like a menu, inventory or in-game HUD. Every element -buttons, labels, fonts, borders, checkboxes, ...- is fully adjustable via a textfile or a "Form Editor". Events like clicking, updating, mousemove or drag/drop can call a Python script that talks with the Engine22 API, or you just attach your own callbacks to them.

Oh, and before I forget, all of this is made on top of some other, deeper modules. "E22_Graphics", "E22_Materials" and "E22_GL" for the viewer, camera, OpenGL wrapping, and shaders. "E22_Input" for the keyboard & mouse, "E22_Util" and "E22_Resources" for basic types and loading DDS images in a background streamer thread. Quite a big pile of code. Although this actual demo program only uses a few functions for the setup & update cycle.



Start the engines
What do we have so far then, besides ideas and dreams? First of all, the engine design as proposed in the "Open Sesame Engine22" pictures (15 March) still stands. A Game Engine is much more than just some classes to render 3D stuff with fancy shaders. Games have to deal with sound, physics, GUI(graphical interfaces), input, threading, resource management, A.I., tools for the artists, and so on. Because of this wide variety, a clear and solid design plan is mandatory. Just stacking up code units to perform X or Y will lead to a tumbling tower sooner or later.

And note the word "Engine". This brings some serious extra complexity. An engine != game (that's programmish for "not the same"). Hobbyists like me tend to create a game, build some libraries, tools and units around it, and call that an engine. With as a result that this "engine" isn't exactly a stand-alone, multi-purpose, flexible, generic product. When one wants to build an engine, it might be a better strategy to produce it isolated from any specific game. Then build one or more games later on, using that engine. Then again it's good to have some goals like "we're going to make a 3D shooter", or a "top-down view isometric racing game". Because making a super-capable-of-everything engine is a somewhat unrealistic goal. You'll likely end up with something that doesn't excel in any area, if you end up with something at all. Ambitions can trigger good things. Too many ambitions will drown you. Big boys like CryEngine or UnrealEngine have the tools to make vehicles or a 2D puzzle. But in the end they are meant for high-end graphics, 3D shooter/action/adventure games. A Command & Conquer alike game is likely better off with a specialized RTS engine.

That doesn't mean an engine is absolutely useless in such a case though. Because an engine itself should have a modular design as well. It's not a single big pile of code that can perform a few tricks. It's a collection of sub-modules and tools that perform specialized tasks. Maybe the Engine22 graphical pipeline isn't very useful for your iPhone Tetris puzzle game. But the lower graphics and sound libraries still might be. The code to play a sound or load an image file can remain the same for numerous applications.

An engine is a multi-purpose toolset. You pick your tools. With that in mind, the engine design should be made. And last but not least, an engine, toolkit, SDK, API, or whatsoever library should make life easier. Writing brilliant code doesn't automatically make useable code for another. Personally, my patience is gone when I'll have to read more than 2 pages or wrestle with vague classes that don't do what I expect them to do. So, when coding anything, I always try to look at it from another perspective. Does it make sense? How would somebody else use it?



Eeny meeny miny moe
As for Engine22, its basic modules can be used for any (3D) application, but the engine as a whole is mainly geared towards medium/high-end 3D PC games. And more specifically for the adventure/action/horror genres. Hmmm, where did we hear that before? Anyway ;) Last months I've made the fundaments of various engine modules, plus a couple of small programs to test them. The most satisfying (and quick) results usually come along with the graphics, thus not a surprise that this portion got a bit more attention. But, I also made a math/vector library, footprints of a multi-threaded system, several file-loaders, and a module that captures keyboard input and such. I won't go into too much detail. Later on when engine modules become downloadable, I’ll explain everything. Going from octrees to pathfinding, from Tessellation to Image Based Lighting. No worries.

During these first "newborn" months, a lot of decisions have to be made as well. Obviously I wrote an "Engine" for Tower22 before, and just doing a big re-write doesn't make a lot of sense if things didn't get improved or changed. My main beef with the "old" engine, asides from obsolete or half-finished techniques that littered the codebase during the last ~6 years, were the lack of consistent modular design, not-so user-friendly tools, and some dated choices programming wise. You'll get a hard time attracting (good) artists if you can't give them a logical workflow with proper tools to work with. And as for the choices...

I often bet on the wrong horse, in the terms of picking a tool, program or platform that gets discontinued. Not necessarily because these tools suck. Heck I dare to say these tools are often above average. But often "my favourites" aren't the public "favourites", and will get trembled by their bigger, more popular competitors. I don't have a crystal ball, and predicting the future is pretty much impossible anyway. But I realize I'll have to be more careful when picking. Stubborn as I am, I probably still don't go with the flow "just because" X is doing it as well, but in the end a project like Engine22 or Tower22 will only succeed if enough people are willing and able to contribute. Being too much dependant on ancient technology will sink the ship.




OpenGL vs DirectX?
OpenGL used to be Goliath, DirectX David. These days it's a bit the opposite. DirectX became mature, and shines in many games. And so it does in examples, tutorials, books and papers. Which is a smart move, because if new students grow up with DX- instead of GL knowledge, you can guess their future choices. GL got stuck in the past with their State Machine approach, old fixed pipeline, and lack of modern features. Some say the days of OpenGL are over...

But, OpenGL made some big moves last years, and pro-GL separatists would say GL is easier to master and multi-platform on top (DX <3 Microsoft hardware). Well, I can't judge really as I never used DirectX. Which is directly a reason to stick with GL for now, although I did consider DirectX. Rebuilding the engine is enough work already, having to reinvent the 3D wheel as well would probably kill me. So, yeah, OpenGL it is. And looking at the recent (version 4.5) techniques, I'm pretty confident DX won't be dead anywhere soon ;) Engine22 will drop all deprecated GL functions (thus no fixed pipeline, all shaders), use VAO's and Instancing, tessellation shaders, and also Bindless Graphics are on the wanted list, which can gain some serious performance boosts.

But just in case I do change my mind, OpenGL is nicely isolated via a wrapper module. Probably it will still suck to replace GL with DX (or vice-versa), but at least the OpenGL calls and "way of thinking" isn't scattered all over the place anymore. The same can be said about other 3rd party libraries that will be used for E22 btw.




Shading languages?
Speaking of betting and horses, "Cg" (C for Graphics - a cross compiler shading language by nVidia) turned into a cadaver. When I started to learn shaders back in 2003 or so, Cg was pretty cool. Tasteful examples from the nVidia videocard bakery, and it worked for both OpenGL and DirectX. But at some point, the samples and documentation dropped dead, and Cg has been announced discontinued not too long ago. Joy. This on itself was a good reason to re-cultivate the engine. What other choices do we have? Not much really. Which is nice sometimes, not having to choose. HLSL for DirectX, GLSL for OpenGL. So, GLSL it is.

I'm not quite sure about Compute Shaders yet though. The main choices are/were nVidia CUDA and OpenCL (not to be confused with OpenGL). Again, CUDA seems to be the more popular kid on school, but I picked OpenCL as I figured AMD video cards may dislike CUDA. Don't know if that is actually true though. Nevertheless, OpenCL worked pretty nice but made a somewhat unfinished impression. I didn't reach the Compute-Shader stage in the E22 rebuild yet, but a re-evaluation is needed. And to make things harder, a third player entered the scene, as GLSL now has a Compute Shader profile as well. Sounds it will integrate nicely with the rest of OpenGL, making it an obvious choice. But as said, got to read some specs first.



Physics
The old engine used the Newton physics library. It's free, has some nice examples, -and also not unimportant for us poor Delphi users-, some Delphi headers to talk with the Newton DLL. Yet I never managed to make the physics “stabile”. And with stabile I mean boxes NOT falling through the floor, and the player NOT getting stuck on the stairs. I guess it’s my own fault, but yet, like OpenCL, Newton never felt finished to me. Which isn't a crime since the guy(s) doing Newton, do it in their free hours, for free (I think). But nevertheless, it is annoying having to adapt on bugs, wait for features, and switch to new versions time to time.

Tower22 doesn't require advanced physics other than stabile player controls, proper collision detection, some tumbling objects, and maybe a ragdoll here and there. Being praised for being very accurate, Newton might be a bit "too much" for Tower22. Then again Engine22 may demand a bit more, just in case you want to destroy worlds or plan vehicles in your game, made with Engine22. So… which physics library is stable, easy to use, not-dead, AND for free? Newton, Bullet, PhysX, ODE... They all seem to be somewhat comparable performance and feature-wise, and also important, still alive and kicking after quite some years. I didn't chose directions on this one yet, but maybe it's smart to at least have a taste from another engine. Bullet for example at least has some cool titles on its portfolio (Red Dead Redemption, GTA IV)...

But moreover, and hopefully you can see the benefits of opening the engine now, it might be a better idea to let somebody with experience in either of those libraries do the coding here. I’m somewhat specialized in graphics, but not in physics really. Are you listening Delphi-physics experts?!


Sound
So far I tried OpenAL (again not to be confused with OpenGL) and FMOD. And preferred the latter, by far. Easy to set up, supporting plenty of file formats, and having cool features that OpenAL doesn't have, such as 3D sound occlusion. The best part is FMOD Designer, an editor that helps you making sound libraries. Which saves me from making yet another editor. Yep, I like FMOD. Easy choice for a change.

However... FMOD isn't free. At least, you can download and play for free, but you'll need a license when using your software for commercial purposes. So honestly, I don't know if I'm even allowed to pack FMOD together with Engine22 (especially when selling the engine). Though if I just offer some supporting code, it might change things. Dunno, yet another thing to sort out.



File-formats & Programs
Asides from programming libraries, an Engine also deals with third party programs. Mainly painting, animating and 3D modelling tools. I prefer the good old version of Paint Shop Pro, and Lightwave. PSP because its light-weight yet pretty powerful, and LW because I found it excellent for quick (indoor) modelling. But not a whole lot people will follow my taste here, plus I'm not a good modeller or digital painter anyway. Most people I know use Photoshop, 3D Max, Blender or Maya. It doesn't really matter though, just as long the files these programs produce can be imported into Engine22. But I'd better make sure at least the common formats are supported:

* Textures --> TGA, BMP, DDS (DXT1/3/5, Cubemaps, uncompressed, ...), where DDS is preferred
* Model import --> OBJ, LWO (lightwave), FBX (AutoDesk & animations), MS3D (Milkshape & animations)
* Audio --> Depends wether FMOD joins the party, but if so: WAV, MP3, OGG and most other common formats

Note that once imported, Engine22 usually converts to its own textual or binary file formats. Assets in the Fuel22 shop will contain both Engine22 files & common formats such as OBJ, Blender, Max, Lightwave or Maya.



Delphi XE
What shall I say about this choice... Would it be the wisest choice? No, it isn't. It's powerful enough for sure, don't be fooled by nitwit-talk. C++ isn't a hundred times faster than Delphi or whatsoever. But simply because the limited userbase, there aren't many tutorials out there, DLL headers for libraries such as GL/Newton/Bullet/FMOD or whatever are often slightly dated or not existent at all (if you read this Sascha Willems, thank you and your mates for maintaining the GL & Newton headers!!), and most likely the amount of people interested in Engine22 will be limited as well. But sometimes you'll have to chose with your heart.

And on top, I bet there are far more C++ based engines out there already, so maybe I shouldn't try to cross their paths anyway. Delphi on the other hand can use some help. Engine22 is not written in Delphi7 anymore though! and probably that's a disappointment for some as well, but we'll have to keep up with the 21th century. And, if there really is a demand, it shouldn't be impossible to make a wrapper and still get it running in D7, but it's certainly not a priority now.


Can't wait to get this working in a real T22 map instead of skybox-testscene!