Sunday, March 15, 2015

Open Sesame Engine22

As you may have caught here, ideas of sharing the engine (not the T22 game!) code popped up in my upper chamber. Erh… why?

Say eight years ago, I would never consider opening the doors of my little magic game factory. Not that I’m that greedy, but c’mon! Me coding my ass of for hours, days, years, and somebody else walking away with it?! A master chief doesn’t reveal his super recipes either, now does he? My code would be my “intellectual property”.

Eight years older and wiser now, or older at least, I’m not that afraid of kids stealing my code anymore. You know why? Because code is just code. Hence, you can legally grab another existing super-engine as well (UDK, CryEngine, Source, Unity, LibGD, …). But that still doesn’t make you a brilliant programmer, nor does it automatically bring you a game. It’s just a tool you can use to realize that game… if you work extremely hard, have awesome ideas, a team of talented people, and some luck that is. A nuclear bomb is still harmless if you don’t know how to arm it.

So, would it really matter if somebody uses my code to create a game? In many cases, it will probably lead nowhere due the lack of willpower or resources. And in the “worst” case, he or she makes an awesome game with it. Well… I can think of bigger punishment. Honestly, it would be my pleasure to see that happen! Besides, what is my “intellectual knowledge” worth if it’s not used for any good? Tower22 won’t be there anywhere soon.

Using someone’s code is something different than stealing someone’s ideas. I’m not giving away the Tower22 game-code, ideas or game contents here (or well, actually we will give some of it away with the planned “Fuel 22” system). Nor does this engine automatically include super-cool shaders that make everything shiny. You still have to climb that mountain yourself, but hopefully such an engine can teach, help or at least reduce the weight. And since the Delphi/Pascal community isn't exactly bulging of such packages and tutorials, this might come on the right place at the right time.

A provisional overview of modules, (external) DLL's and other programs to Engine22. Should be enough to make a game :)

But again, why would I release my code? Where is the catch? Helping poor programmers out there is very noble, but not the real goal, is it? Indeed, I’m not a generous Delphi Jesus. No, I want something in return: attention for the project & a better engine. Allow me to explain.

Besides that little piece of elder wisdom, maybe work twisted my perspective a bit. I used to write everything myself. No other programmers in a circle of 3 kilometres. or I would raise my tail and moan like a street cat. This gave me freedom and some status at work. But at the same time, it's lonely at my tiny self-created top. Work last years forced me more into collaboration though. Lack of time to finish everything on your own, plus I got a more supporting, advising and teaching role towards other programmers. Hence, we finally hired a second programmer at one of my jobs, and frankly this new role suits me. It's not so bad to share.

Real-life example of a generic code Framework
To illustrate. The majority of my work consists of programming harvesting vehicles. Hydraulic controllers for big monsters. If you thought about hillbillies, chewing tobacco, potatoes, marrying with a pig, mud and manure, well, you are partially right. But don't underestimate the technological finesse behind the layers of dirt on that combine machinery. Farming isn't about rusty forks and bronze-age tools anymore. Self-propelled fully automatic vehicles, GPS, Telematics, drones & geo-mapping, state-of-the-art diagnostic tools, heavy duty big-ass diesel-engines, virtual terminals. Just a grasp.

My job is mainly about making the hydraulic controller computers (a bit like PLC's), (touch)displays, and the (CAN-bus) communication between those computers. Though more and more side-interfaces are entering the area as well. People want to monitor their machine fleets on a laptop, walk between machines and download performance-data or problem-codes on their iPad via WiFi, exchange data between machines and containers that go to food processing plants. My work is also about presenting all this complicated high-tec stuff as simple as possible to both end-users and service technicians that often don't have a computer background at all. Eazy doez it.

Anyhow, plenty to do, and obviously I can't do everything on my own. Certainly not now we teamed up with English, French and American machine builders as well. This group of manufacturers brought new challenges; how to share technology, rather than re-inventing the wheel over and over again? Of course each branch could keep programming their own stuff their own way, but why not trying to team up, share code, make standardized communication protocols, and create a "common feel" so that technicians from companyA will understand the computers from companyB as well?

So, one of my tasks became making a Display Framework for the touchscreens in our cabins. Although the "front" -the workscreens with gauges, buttons and meters-, differ for each machine, the heart and advanced features are the same on all our touchscreens now. I made various generic C++ modules such as math libraries, Linux platform basics, CAN-bus systems including J1939, a HTTP server, graphical libraries, standard screens, diagnostics, databases + editors to define everything, and so on. The Lego bricks for building your on-board vehicle touchscreen.

Sounds like a logical thing to do, but doing this properly was/is one son of a bitch. Because I wasn't alone this time, I had to make sure that everyone (read other programmers in our group) agreed, liked and especially understood this framework. That goes for both experienced and "beginner" programmers. I had to make sure the Framework would help the guys realizing their software by making things easy and logical. Some of the programmers aren't really programmers as they learned this themselves at work, and never saw a line of C++ before (the most harsh language of all, my goodness). This truly forced me to think twice, thrice about the design. Every module, every function name, every trick had to make sense, and should be pulled off with a minimum amount of coding, and in consistent ways to avoid confusion.

Then on the other side of the boxing ring were the more experienced programmers, machine designer veterans. Ask two persons to design a computer program, and you get 2 entirely different systems. Both visuals and technical bits behind the scenes will be done differently. And you know what happens if two experienced programmers want things their way: clash! The Framework had to make sense, gain endorsement, but also had to remain flexible instead of dictating another what to do. If X wants to make flying saucers, discarding pieces of my "standard chunks", tweak the entire lay-out, or add a complete new feature, the framework shouldn't be a restriction.

After a lot of thinking, discussing, coding and trying, I think the end result is pretty impressive. It's not a game-engine, but the same problems regarding flexibility, performance, and clarity arised. An Engine might look good if it works for 1 project, but does it still work for 10 different projects, and is it robust enough to survive future add-ons/changes, such new communication protocols? Needless to say, this whole experience is valuable knowledge when (re)writing a game engine such as the one used for Tower22. If I had to do it all over... I would definitely do a better job. But it's not just the technical challenge. "My stuff" being used by others now, opened a new dimension of teaching people and writing Wiki articles, as well as listening and taking a step back if needed. Brilliant code or not, it's still worthless if others can't / don't want to use it.

Benefits of an open engine
Nice personal story, but what has this to do with Tower22? Well, why not do the same here? If I put my code (read the engine, not the whole game!!) available for you Delphi fanatics, it will force me to do a better job. Game-programmers will probably recognize the “quick & dirty” way of achieving things. Intentions are usually good, but sooner or later we want results. Showing off cool graphics or playing your first level. This is in contradiction with what writing an engine is about, focussing on multi-purpose / generic / universal frameworks and tools or editors. In other C++ words; writing engines != writing games, writing engines == boring.

Quickly trying to get your first level started, usually means a dirty hack or bypass in your engine. Nice for short term results, but making an irreparable mess on the longer term. Technically, many hobby-engines aren’t really multi-purpose engines but merely a bunch of class-libraries written around one specific (game)project. And as for supporting examples, consistent design and solid documentation? Hahaha. Sniff.

But if I promise to share this engine with others –thus for other (maybe non-game) projects- it will force me to keep it on the right track, and to dodge the temptation of getting my own stuff done quick & dirty. And if I want others to use it, I also have to spend some energy on making (basic) examples, documentation, and (video) tutorials. And last but not least, the open character of the code will allow others to track bugs, request useful features, or even help me writing additional modules, editors, examples or articles. Though I’ll have to note this won’t be a pure open-source project. Yes the code is open, but I’ll maintain the code myself because A: I’d like to guard the quality, B: no time (yet) to process code-commitments on a bigger scale, and C: I just like doing it myself. Nevertheless, specific modules or tools might be outsourced to others in case I lack the experience or time to do so. Which is yet another good reason to have your engine well designed and modular. At this point, getting help on Tower22 is somewhat impossible as it's 1 gigantic pile of (messy) code really.

Combined with the "Fuel22" asset store idea, Tower22 will hopefully get a kick in the right direction with active help from the community. Whether you buy (and thus support our artists) our 3D props, report stinky code bugs when using the engine for your own projects, make example progs, write Wiki tutorials, or just spread the word; it will all help the T22 project. Power to the people.

Roadmap to success?

What’s the plan doc? Well, before anything really happens, I have to get the engine in shape first. Read a re-write (to Delphi XE). That sounds like an extremely lengthy job, but the good news is that I already know how to do it (better) with the experience from the past. In general when programming, most time gets spend on research, googling, thinking about designs, (re-re-re)programming, and of course debugging / testing. Research has been done already (for a big part at least), and the design is pretty clear in my upper chamber. Many of the code snippets and tools can be re-used as well. And as said, I’m a plow-horse. Not a brilliant programmer, but I can certainly push the tempo.

The engine doesn’t have to be *finished* before releasing it. Software is never finished anyway, and the modular design allows to release sub-modules as stand-alone packages. For example, a sound module doesn’t need the entire engine to be up and running. The whole point of modular design is that you can make a sub-selection out of the modules you really need, and discard or replace the others. The more abstract modules will be released first likely, and then we climb up the ladder towards the more game-specific and complicated modules. How long it would take? Hard to say, but believe me, I managed to write quite a lot in just a few weeks (with a crying baby on the background, for that matter)!

In the meanwhile, the Fuel22 system and Tower22 game (demo’s) will continue just as well. As usual, the demo’s will be used later on to get attention for the project. The biggest challenge is to migrate the existing code step-by-step towards the new engine. It’s like restyling your house. Furniture gets thrown out and replaced. One-by-one eventually, although at some point all your stuff has to be placed outside when putting a new floor or painting the walls. The target is to keep the Tower22 game “online” as much as possible, so it doesn’t have to wait on the new engine. Remember, we still have an official demo to release as well!

The Fuel22 website will also require some more months at least. The aim is to have it online just before our next official demo movie is finished, so that new artists (that will hopefully join after seeing the movie) can work & get paid via the Fuel22 system. If you missed Fuel22; it’s a webshop to fuel the creation of assets we need for a next playable demo. Instead of throwing a Kickstarter campaign we’ll sell a part of the stuff we make (think about 3D objects, textures, drawings, audio FX, …), and the money goes straight to the artist so they get more reason to join & get motivated to keep producing. Something this project as a whole really, really needs.

The same Fuel22 system will also be used to both download the free engine modules, and later on to buy the "Full package", when it's ready. Selling the engine?! Yes. The idea is to release a “lite” version for free –the stuff we discussed above-, and a “full” version for a low price. This full version would contain some more modules, the graphical pipeline + advanced shaders we’ll be using in Tower22, and editors. Obviously we’re still far away from that, but again the idea is to fuel our project by selling whatever we can. So why not selling the engine as well? You'll be happy with a cheap engine, we'll be happy with a donation that can be spend on rewarding the artists for their 3D/mapping/audio/drawing/whatever work, and me happy as all those years of programming finally gets me something return. Maybe even the girl will be happy after all those lost hours then ;)

Well, more information will certainly follow. That also means this blog may get more engine-programming related articles. Showing examples, engine design, et cetera. Does that sound like a plan, A-Team?!