Sunday, March 19, 2017

Save Load Save

1 Continue Left
Don't know how old you are, but when I was your age, we used to play games with our bare hands. No health recharge, no "switch-to-easy?" after dying 3 times, no IDDQD, and no AutoSaves. Hell, no saves at all!

Hard to imagine, but indeed some of the old PC, NES or even later SNES games I played, didn't give you the opportunity to save, switch off the system, relax, and go play outside. No sir. Finish what you started, dammit. Countless times being in the very last level, finally, nerves up to the throat, and then mom yelling "bedtime! NOW!". Jesus, now what?! Restarting was not an option, so we had to turn off the TV but leave the system on, sleep, go to school, and hope mom didn't turn off the system (and hope the Nintendo wouldn't catch fire). And of course, next day, 16 hours later, we died over some ridiculous jump or impossible end-boss anyhow. Some games showed some mercy and would let you restart the level, but mostly you just had to replay the whole fucking game once the “Continues” depleted. Developers were soulless beings. Or maybe they were just too stupid to make a SaveState back then...

Well, as a cold comfort, most games weren't that long those days. If you include all the dying, restarts, flying gamepads, swearing and berserk kids, a game could easily take days or weeks to finish. But once you knew the trick, the average game could be finished within a few hours or less. And the somewhat better games usually had a Password system though. Or at the very least, gave you some cheats to skip sections, like the Warpzones or Magic Flutes in Super Mario Bros.
 Axe, Axe, Heart, Turd, Heart to head on Dracula's castle.

The Hard-coded magical Flute
But that was then, this is now. Glad as I am that the Tower22 playable demo finally has enough content/length that saving progress would be a nice thing, I certainly don't want to play a goddamn flute to skip parts.

“Saving & Loading” is probably not the first article you will find when learning about "Game Programming". It's one of those "yeah, whatever. Some day later I'll bother." features. Usually I just hard-coded some cheats or hacks to start elsewhere in the game, magically unlock doors, or give me item-X. That may work fairly well for level based games, but Tower22 is just one big world. And to make things worse, it's sort of a puzzle game. So a lot of conditions and events depend on stuff that has been collected or triggered earlier on. Complex "chains" like these quickly get tedious to overrule with hard-code, so basically I just had to replay the whole game to test if Script-X at the end of the demo works... which didn't because of 1 typo in the LUA script. 0 ups, game-over restart. Just like the good old days.

Yes, about time for a Save/Load system. But honestly, I never wrote a Save/Load before. Well, I did for other programs, but not games specifically. Usually the games I wrote weren't really playable, or had less than one level content ;) So what I'm about to write might not be the most solid solution, then again I'm guessing there is no universal standard answer on this topic anyway.

Thankfully, I sort of anticipated on this early on, so the engine had some tricks and mechanisms prepared. Which sure wasn't a bad idea, because I think for a more complicated (puzzle)game like this, where a lot of various stuff has to be saved in order to pick-up, it's actually quite a fundamental part of the engine entity systems, filestreams, and so on.
Save – Just a little bit
Correct me if I forget something brilliant, but I'd say there are three ways to deal with saving to begin with. Option1 is pretty simple: just save some numbers like the current level, and remaining lives (called "ups" back in the Good 'ol days). It's enough information to get us started in level-X. But what if your levels are huge, or if you don't have levels at all?

In that case you probably want to save a bit more information. Option2 does the same thing, but in more detail. You would save details like the actual player position, unlocked doors, his inventory, whether collectable items were picked yes/no, and maybe the status of your enemies (if & where they got(killed)). Now you can resume without having to restart, re-collect, or re-kill all your opponents.

Option2 is still limited though, and requires a good portion of very specific code. What needs to be saved, and what doesn't? And I'm talking about killing... but maybe you were making fishing game or something. When making engines, you don't (shouldn't) know such details, meaning you'll have to offer a more abstract solution.

Option3 is pretty hard-core: you literally dump the entire state of your world/level and every entity inside. All exact positions, bullet holes, destructed environment, caught fish, and so on. Works pretty well if your levels aren't too big (or if there is just very little you can alter), and is often used for shooters and such. And talking about old shitty games that couldn’t be saved, an Emulator can! It dumps the exact memory(“RAM”) state. However, SNES RAM was about 128 kb, thus resulting in tiny files as well. But don’t try this at home with a modern game, that may eat 3+ GB of RAM!

Oh, and I forgot option4: just don't save at all. Easy Peasy. But seriously, in case you are still baffled the devs didn't give proper "Save" options back then; old systems like the Gameboy or NES gave you a bunch of (kilo)bytes on a cartridge, go figure.

What the hell is that?! That, my grandson, is a SNES game. Didn't have Steam to download games, even CD-ROMs were yet to be born. Games looked more like computers back then, having a ROM chip and their own (battery backed) SRAM to store savegames. I always wondered why some games were almost 30 dollars more expensive than others. Well, as you can see, some games actually contained more physical hardware, like extra RAM memory or a SuperFX chip to render 3D stuff. So if your game didn't let you save anything at all, it was probably because those cheap assholes saved on a SRAM chip.

Save - anything Dynamic
I think modern games use option2 in many cases. But instead of saving a bunch of special numbers, it saves the attributes (position, state, special properties, …) of anything Dynamic – stuff that can change. Static stuff – stuff you can reload from its original source, such as sounds, textures or the World Geometry (unless destructible) can be ruled out.

The cool thing is that you can do this in the same manner as your Map-Editors would save the files… but preferably without the map geometry itself. Why? Well, if your mapfile is 500 MB of "level data", a SaveGame file would be equally big. Now storage space isn't our biggest enemy anymore, but it takes quite a while to load all that, and these days you may want to store these files in a Cloud as well, which still makes them (too) big. And don’t forget, a modern game saves your progress every 30 seconds or so. Don’t want to send those poor kids more than a minute back in time, if they gang-raped by a horde of demons again!
Yellow stuff = static data and/or initial states of dynamic data. Blue stuff = computed on the go, green stuff = data that can potentially change, and therefore has to be stored in a SaveGame file.

The Cleaning Crew
Another less technical issue then. What IF everything would be stored "as it is" in a game like GTA? The whole city would turn into a gigantic wrecking yard! Bumped lampposts, burned cars, crashed helicopters, busted hoes. Sounds pretty cool, but what if a trainwreck blocks your way into the building your next mission needs you to be? Exactly, like in the Good 'ol days, you can suck it and restart the whole game. And no, unlike SNES platformers, GTA doesn't take 2 hours to complete.

Another example, just for the fun of it, what IF you would shoot all space-crabs, flying lava dragons, and other insect-scum in a Metroid-like Exploration game? The game would become pretty boring, and in case of Metroid, even more lonesome once you killed all life on the planet!

No, that magical cleaning crew that repairs the lampposts, cleans the streets, replaces the cars and clears homicide scenes are actually quite welcome. What really happens though, you walk/drive/fly far enough from a certain spot, so it becomes "unloaded". Then when returning, the original scene -as originally created in the MapEditor- gets reloaded again. Clear from mistakes, forgiving your sins.
Please clean-up the Millenium Falcon on Isle Five.
Local versus Global entities
Tower22 does the same. You can run over a porcelain vase, but the cleaning ladies are kind enough to clean up the mess and place a new one - if you are not looking, they're shy. However, some entities should stay as they were. Unlocked doors should remain unlocked typically, collected items should stay gone, solved puzzles should stay solved. And Bosses should stay dead, definitely that.

In Engine22, when making maps in the editor, every entity (furniture, walls, items, monsters, lights, particles, and so on) can be marked as "Global" or "Local". Local entities are bound to the Sector they’re placed in, which is typically the room or corridor it stands in. When (re)loading that piece of the map, it will also load all of its local entities back in their original state (and yeah, a script can still move/hide/alter them afterwards, if needed).

Global entities on the other hand aren't part of a specific Sector. Global entities can eventually move throughout the world, for example, being carried as an item by the player. Global entities can also remain stationary, but store a certain state or other variables. Under the hood, each entity can have a property-list, or "Blackboard" as we call them now in BehaviorTree terms. This Blackboard is just a bunch of keys + values, like "doorLocked := false", "fuelLevel := 45", or "puzzleCompleted := true". Local entities can also have a blackboard, but as said, its properties will reset back to default once being reloaded.

Global entities are not saved in Sector(map-piece) files, but in 1 global file... which is pretty much the "SaveGame". When starting a new game, you actually also load a game, but filled with initial states. Then when moving on, you alter that data, and save it under a different name: your SaveGame, Slot, or whatever you would like to call it.


SaveGame file content
So résumé, how does the "Save" file actually looks? In my case it’s a binary file, with all global entities stored. This is done in the exact same way as the MapEditor would do. Entity classes have their own "readFromStream" / "writeToStream" functions and may differ in detail, but in general they would store a matrix, references like which material, sound or objectData has been assigned, its current state, eventually defined by a Blackboard (property-list). Speaking of which, there is also a Global Blackboard stored, which contains all global "game variables", that can be shared amongst all entities. Also handy, entities can also refer to each other. For example, entities like the Player or a storage trunk can have an "Inventory", which is basically a collection of items, thus pointers to other entities.

Last but not least, the engine uses a callback to let you -the Game- write custom data. In the case of Tower22 that would be the clock/calendar (which day are we, how late?) and additional non-engine attributes related to the player.

It works, though I must warn about the snakes and scorpions in this method. First of all, it’s a binary file. The good part is that ordinary teenage pukes can't mess around with those files that easily, and binary files are relative small compared to text-based files such as XML. They also load a whole lot faster. You’re not parsing stuff, you just suck stuff straight into RAM objects / arrays.

Really, I always wonder what the hell games or other programs are waiting for when hitting "START". When I click the RUN arrow in Delphi, I'm playing Tower22 4 seconds later. And yes, hundreds of (texture, 3D, material, audio, world geometry, ...) files have been loaded then. I dare to say modern software has become extremely lazy when it comes to RAM usage, file-loading, and so on. My laptop has 16 times more memory and CPU than ten years ago... but Apps also have become 16 times shittier in terms of resource management. Conclusion, everything still sucks.

So yes, I'm a binary fan. But there is one BIG BUT; they break easily. Just 1 wonky bit is enough to screw up the whole loading process, and giving you a headache trying to fix the file. Now if your binary format is final, and if you can guarantee the save-process doesn't make mistakes, binaries are your friend. Otherwise, expect trouble. Here a possible scenario. You finished your Minecraft game and sold 800 billion copies. Fine and dandy, but now the community cries for a new game element: an attribute that describes if a cube stinks or not. Being binary, you can't just add another boolean in your per-cube properties. Older versions wouldn't understand this file-format and basically read too little bytes per cube, newer versions won't understand older files, and basically read too many bytes per cube. Making the program crash.

You forgot to add some sort of "version" identifier prior to reading your cube-data, with as a result hundred millions of enraged teenagers. Kids lost their 8th-world-Minecraft-wonders, jumped in front of trains. You lost all your money, your wife ran off with that big black dude, and your mother hates you too.

That's one possible scenario. Now you could have avoided this by adding a version or something, so that your program can switch between 2 reading strategies, but what if you didn't have version numbers at all in your original files? Use them. Always. In my case, every (sub)instance that contributes to a FileStream, first writes a version byte. It could be that 99,9% of the contributors didn't change, but one specific entity type did, so I only have to alter its write/read procedure based on its own subversion.

Still, more structural adjustments can be a pain in the ass, and if you have a bug in your filewriter, it’s very hard to fix corrupted files. Think about those raged teenagers.

This is a higher level loading procedure; a Sector (map part) reading all its sub-entities (walls, floors, lights, eplosive barrels, sprites, ...). Note it first reads a version byte, which tells me how to anticipate on what follows. If something would ever change, I can "if ( version < 10 ) then ...". Entities on their turn do the same thing, inside their "ent.loadFromStream" procedures.

Incompatible Save states
Speaking of versions. Another problem my system has, are possible differences between dated SaveGames and newer versions of the game. In the playable demo, you have to collect trash at some point (not kidding). But I forgot to mark that trash as "Global" entities. So if you reload the game, all the trash would be back, even if you already collected them. Which was actually a nice bug, as you could cheat and reach the "18 items-collected" target easily. So, I fixed that by marking the cola cans, banana peels and blood-vomit pools as Global in the MapEditor. But, now all trash was suddenly gone. It wasn't stored in my local sector files anymore obviously, but neither in my SaveGame. Result? Couldn't collect all trash & finish the puzzle. The only fix was to restart the whole fucking game - like in the Good 'ol days. Doh.

I don't really know how to fix that -easily, without writing crazy hard-coded patches in the load procedure. One more sophisticated method might be not to simply dump all Global Entities into a file, but to write their "Mutations". So you would always load the initial "original state" file first, then go through a list of alterations. EntityX moved from A to B, Trash entity #123 deleted, Poor Monster's health changed from 100 to -1, and so on. That would at least respawn the trash. Heck, you could actually rewind all your actions like an UNDO function!

Yet, there is still no guarantee that this works. The initial state might be different in a newer version of the game, leading to certain events that weren’t covered in your dated file. You can’t reproduce the same state if certain actors were missing in your “flight-record” data. Plus I'm guessing this mutation-list gets veeery long, veeery quick. Basically you can’t just change the puzzles, then expect your SaveGame still works well.
Slept well Samus? Metroid would save your inventory, which bosses are dead, and which passages have been revealed, I think. But you could get locked if the map or some of those passages would be changed in a game-update... Good thing there were no patches back in 1994. Devs had only one shot to make their game work well. Good 'ol days.

The thin red line
Probably a better idea is not letting your game care about every single detail, and having a fall-back plan. In Tower22, you can't save while playing really. You save at special spots - in bed, when sleeping, to be more precise. So a puzzle like this trash-collecting-nonsense is either finished or not, don't care about the details. Tower22 doesn't have levels, but storing the current "Day" may reveal enough information to estimate what has been completed so far. A game like GTA doesn't save mission-details either, it just saves whether missionX has been done or not. Based on that, the game can decide what your next assignments and phone-calls should be, rolling on.

So in other words, it's probably a good idea to have a "Life Line" stored, telling the global game progress. Like "Chapter 6". In the very worst case, you can help your players by letting them return to a fixed state based on that, not having to replay the whole game - like in the Good 'ol days.