Lessons learned from writing an 8 bit game engine from scratch
Since we decided to write our game engine from scratch, I thought I'd outline some of our design choices, and the way we do work and iterate in this post. This post was written by our developer, Eric Lavesson.
This is going to be a slightly odd news post, and pretty much a developer-centric one. I wanted to share some of the lessons I've learnt from developing Cathedral for the last 8 months, and hopefully there's some nugget of wisdom in here that might be of use to some fellow game developer(s).
In this post, I thought that I would give a better picture of who I am, and describe some of the technical aspects of our game a bit more in-depth and how I've been working on the source code so far. It's been an interesting journey, since we decided to write the game engine and all tooling from scratch.
So, let me start with a small piece of background info about myself and why I decided to do this, and let's move onto some of the architectural choices of Cathedral.
Who am I?
My name is Eric Lavesson - I've been hacking code since about the age of 9 (which is a bit more than 20 years ago). I started out writing text adventures on an Atari 600XL that my parents bought me in the late 80s. I love how these old computers just threw you right into a BASIC prompt when starting it up! About the same time, I also got my hands on my first NES, allowing me to play games that would turn out to be some of the most fun games I've ever played.
At this point, I really wanted to write even more games, so I ended up getting a bunch of newer Ataris over the course of some years. Long story short: I basically went from having an Atari ST, to a 520STFM to a 1040STE before getting my first PC. During this time, I learned how sprites worked and a lot of the basics of game design. I wrote the occasional Mario clone and a few 2D shooters in a language called STOS, which was more or less a dialect of BASIC targeted towards game development. The Amiga had a similar thing called AMOS.
Anyway. After my last Atari, I started writing some QBasic on PC, followed by starting to look into C and some really simple assembler for DOS games. I remember a lot of the old games for DOS quite fondly too (Commander Keen, anyone? Or Duke Nukem, pre-3D?)
As I grew older, I started writing more and more code in C++, and started looking into DirectDraw to start with. I later went from C++ to Java and finally C#, which is the language I do most of my dayjob-related consultant stuff in. Even though I don't really work as a game developer, games is what got me into programming all those years ago. It's the kind of thing that you don't forget about, and it's a passion which is hard to let go - even through all those years of never really thinking about releasing a commercial game.
Today, I'm working together with Aron on our first commercial release, Cathedral. Inspired by all those awesome games we both played when we were younger.
So why a custom engine?
About ten years ago, I started writing a bunch of D3D stuff, first in C# and then in C++ and wrote a small game engine. I've written numerous smaller games and engines in D3D, and decided to switch to OpenGL a while back to learn that API instead.
When I discuss game development with people and they hear that we're building the game engine and all tooling for the game ourselves, people often stop to ask me why.
This is in no way a bad question - In retrospect, it sometimes seems a bit insane to write it from scratch, especially since we're doing cross-platform development, trying to get the game to constantly build on Linux, Windows and OSX. There's a lot of engines that can do this for you.
I don't really have a better answer than "Because it's fun" to be honest. I could pull out all kinds of arguments about how writing a game from scratch in C++ gives me full control over my code, and how it *must* be a lot more performant. But in the end, it's mainly because it's fun, and because you learn a lot of stuff when you do something like this.
On to the technical side - The game engine is written in C++. We use CMake to generate project files on each platform/compiler (right now, we got a working build for MSVC and GCC, and we're working on building on Clang as well). Another REALLY important part for me, is the use of a sane piece of version history software (in my case, Git). It's not just about having backups, but actually being able to track changes, find bugs when they occured at some unknown point in time etc.
CMake helps us generate everything we need, such as build files for the game plus all dependencies (which are stored as git submodules with their own CMake files). After this, we run a series of Python scripts to generate a bunch of JSON files with resources and mappings (BSON in debug builds) as well as copying in all the assets for the game (Yes, you generally want to keep these in a separate repository, at least if you're on Git. Trust me.)
The game also makes use of a bunch of JSON files for levels (and again, BSON in release/optimized builds). I wrote an editor in C# and WPF for this:
The editor is basically never done. It gets changed as the game needs it to change (which happens every now and then). When tools like this reaches a point where it's actually useful in practice, it's extremely nice to have a tool which is tailored to your games' exact specifications.
When everything is generated, it's time to build all the small pieces that make up the game itself...
Some tips about software architectures
Architecture is one of those things in software that's always hard. Building a mess is easy, and most human beings tend to not handle highly complex domains in software all that well. By finding the right boundaries in software and decent abstraction levels, it helps us understand how each piece of the game fits into the engine without having to understand absolutely everything at the same time.
There's a giant challenge in actually finding these boundaries and abstraction levels without putting them in the wrong place. The thing that has worked for me/us (and that I hope might inspire someone else), is the following ones:
Write prototypes. Lots of prototypes. They let you see your game for what it really is, without all the bells and whistles. It also lets you throw away the bad ideas early on an keep the good ones. You can always build more robust versions of stuff later on.
Iterate early and often. If you're not releasing your game publicly, then at least release an internal build to other team members as often as possible. If you're working on a game yourself - then get some friends or family to play it instead. Collect feedback and incorporate the good parts into your game.
Don't be afraid to throw code away. If it turns out that your code is cumbersome to maintain (like, having to add an argument several levels deep throughout a call chain), don't keep bashing your head against that code. Roll it back and reimplement. The next time will be faster, and you now have a much better understanding of your game's code anyway. Code isn't sacred just because it's there. (By the way, this is where a version control system REALLY shines)
Try to do something like test-driven development or some other form of development that forces you to design from the outside and in. It helps you find and refactor your code to more and more reasonable levels of abstraction. Yes - contrary to popular belief, game logic can be unit tested. It actually forces you to do a clean separation of presentation and logic when you do. In our case, we're using boost.test in conjunction with TurtleMock for mocking parts of our game engine during test cases.
And by the way, when I say "write prototypes", I really, really mean it. I can't stress the importance enough. It's going to give you wonderful insights. As an example, this is what the first version of Cathedral looked like:
For reference: The blue rectangle to the left is the player. The red blocks are the ground. Bluish pond to the right is blocks that will hurt you (which later turned into spikes in the game).
The different modules of the game engine
Finally, I thought I'd mention what we ended up actually using in our game engine.
We're using OpenGL for rendering. To get around cross-platform context creation and everything regarding that, we're using GLFW3. This also gives us controller and keyboard support out of the box. The renderer is built around a Quad tree to allow some serious culling, meaning that we don't have to care too much about levels getting too large anytime soon. Adding more elements to the level often only affects the framerate slightly.
For sound, we selected OpenAL. I did this by using a wrapper API around OpenAL called cAudio. We did run into some problems with CMake here when using cAudio as a submodule, but luckily we were able to patch it and get the pull request accepted.
The game is scripted in Lua. I was thinking Python first. Mostly because I kind of like Python. However, it turned out to be less than trivial to implement, and after a lengthy discussion on TIGSource, I changed my mind to Lua.
The physics engine is a completely custom one. This one REALLY determines the feel of the game, and getting the controllers to respond exactly the way we want them to has been a high priority. I considered using something like Box2D at first, but decided against it, and ended up writing my own. And actually, this is a really nice practice if you want to understand your game even better.
I hope this read was interesting for other developers and game designers too, and that you got something interesting to take away. As we continue to build on Cathedral, we constantly change and update both the source code and our understanding of the game itself.
Feel free to send me a PM or comment in the post directly if you have any questions.
// Eric Lavesson
PS: Writing your own physics engine also means getting hilarious bugs. It's worth it. If nothing else, it's a good laugh and a days work wasted.