Here we go again... (porting Cthangband, to Godot this time!)

[Dean Anderson]

So, progress is slow and steady.

Looking purely at the numbers, the Godot project has now reached 25,000 lines of GDScript. In theory that means I'm about 20% of the way through (the C# version has 125,000 lines of code), but in practice I'm a lot further than that because C# tends to take more lines of code to do something than GDScript does. So I'd guess that I'm actually around a third of the way through.

When it comes to feature completeness, Here's where I am:

• Basic display and input: DONE
• Menus and loading/saving games: DONE
• Character creation: DONE
• Procedural level generation for towns, wilderness, dungeon levels, and caves: DONE
• Item and monster randomisation and generation: DONE
• Time keeping and speed/turn structure: DONE
• Character status from buffs/debuffs/equipment: DONE
• Character walking/running around, and level transitions: DONE
• Character death/game-over/scores: DONE
• Quest levels: DONE
• Interaction with items (pick-up, drop, wield, take-off): DONE
• Interaction with environment (doors, traps, digging): IN PROGRESS
• Monster movement:
• Melee combat:
• Target/Look system:
• Ranged combat:
• Spellcasting:
• Mutations:
• Ancestral powers:
• Item activation powers:
• Monster powers:

That makes it look as if I'm farther along than I am, though, because a lot of the things I've done are quite small and some of the ones left to do are huge.

For example the monster AI is a massive task, and while each individual spell/mutation/power is usually straightforward there are just so many of them. For example, there are 96 mutations and 256 spells, and that's before we even get to item powers, ancestral powers, and monster powers. Even if each of those only takes 10 minutes to port (and that's very optimistic assumption), we're looking at around 80-100 hours - and since I'm doing this in my spare time, that's quite a few weeks.

So realistically I'm still months away from finishing, but at least I have the basic framework of the game up and running so I can see progress as it's happening.

[Dean Anderson]

So, another quick update...

I switched from having everything run on one thread using the await paradigm to manage waiting for user input to using separate threads for the user interface and the game logic and using the mutex paradigm to manage them.

Basically, I have a mutex that I use to control access to the data that both threads use (the keyboard buffer and the screen text cells) to make sure that the threads don't both try to access the same data at the same time, and when the game thread wants to get the next key from the keyboard buffer and there isn't one for it to get I use a semaphore to block the thread until the ui thread has put a new key there.

Thankfully, Godot makes that very easy to use, and I can still use proper debugging and breakpoints in the game thread.

To my surprise, this has vastly improved performance. It seems that having a couple of hundred "await" commands every frame was the main bottleneck, rather than the interpreted nature of GDScript. So I'm continuing with GDScript for the moment.

The other performance issue I was worrying about was the level generation, but it occurred to me that I had only implemented the "open cavern" style of dungeon level, not the traditional room-and-corridor style. Open caverns are much slower to generate than normal levels because they use a 2d perlin noise field in the background to determine which grids should be rock and which should be open. Given the calculations that need to be performed on every grid tile in the level, this is very slow. Normally this doesn't matter, because only around 1-in-25 levels will be an open cavern, so even if it tries to create the full 100 auto-scum levels before giving up, only four of them will (on average) be cavern levels. But because they were the only ones implemented, it was trying to use them every time.

I'm in the process of porting over the normal room-and-corridor level generation code now, and hopefully that should be a lot quicker - quick enough that I won't need to switch language.

[Dean Anderson]

So, it's been another couple of weeks and my Godot port of Cthangband is up to around 20,000 lines of code. You can now move around the town and wilderness, and even go down into a dungeon level and walk around, although the monsters currently just sit there and don't do anything. But I've got the core of the game working. Most of the remaining 50,000+ lines of code are all the special things like spells and monster abilities and item abilities and so forth...

...and unfortunately it seems to be unplayable.

The problem is just the performance. The way Godot is written, which is ideal for its target audience, is that it has a heavily optimised graphics, sound, and physics engine under the hood and then an interpreted scripting language sitting on top.

The fact that the scripting language is interpreted isn't an issue for most games, because the scripting and logic is a tiny fraction of the runtime of the game compared to throwing graphics and physics-objects around. Unfortunately, Cthangband (and Angband in general) is the very opposite of that. I'm not using the physics engine at all. The only graphics I'm using are a bunch of characters and a background image (and occasionally a software cursor), and the program spends the vast majority of its time processing logic and code.

And the interpreted GDScript is just too slow for that.

Simply moving around the map (even without any monsters acting) has trouble keeping up with your keypresses, and when going downstairs to the first dungeon level there is a ten to fifteen second pause before it finishes generating a map. Admittedly that's with the "auto-scum" option on so it's actually generating 100 maps looking for a good one and failing to find one, but it was still never that slow in the previous versions a few years ago so there's no way I could release a new version with this performance.

So I've a horrible feeling that I'm going to have to re-start the project, but using the C# extensions to Godot this time. That's more fiddly to manage (the whole point of GDScript is its ease of use) and pretty much requires me to use an external editor instead of Godot's IDE, but it means that all the non-display code (i.e., 95% of the code) will be running at the speed of C# rather than at the speed of interpreted GDScript.

I'm a bit reluctant to throw away a couple of months worth of work, but I think it might be the way forward.

[Dean Anderson]

It's been a couple of weeks, and I've got a lot further, to the point that I'm almost ready to actually start implementing the main loop.

The "depth-first" approach meant that the journey between "town creation is done" and "display the player in town" turned out to be quite an adventure, because to draw the screen I had to implement the whole of the view updating and line-of-sight code, and then to allow you to have some starting townsfolk in the town with you I had to dive into the monster spawning code, which might not sound that bad to just spawn some townsfolk, but the same code also has to be able to deal with spawning groups of monsters or spawning uniques with escorting groups or spawning monsters in a dungeon that's biased towards a particular type or spawning monsters based on a summoning spell/ability.

So in the end, going from "I've created a town" to "You can see the town" turned out to be another couple of thousand lines of code and meant that I'd spent three weeks porting stuff before being able to see if it worked (spoiler alert: It didn't, and it took all weekend to track down the bugs - at one point I had to dig out the old Cthangband 3.1.0 code to make sure that the C# of Cthangband 6.x was doing what it was supposed to do).

One interesting thing I'm seeing in switching from C# to a Python-like language is how much it's compacting things. GDScript functions seem to be ending up somewhere between half and two thirds of the length of their C# equivalents. A lot of this is because, like Python, GDScript uses indentation as syntax rather than using brackets. This means that the C# code:

if (foo)
{
   bar;
}

becomes:

if foo:
   bar

So four lines of code have become two. It's not just on if statements that you get this, but on almost every code structure, and that's why the code is shrinking. I reckon that by the time I've finished the port, the 125,000 lines of code will have shrunk to something like 75,000.

But last night I got it correctly displaying the town around the player, complete with proper view and line of sight and townsfolk. So now I just have to do the rest of the display around that (stats, etc.) and I'm ready to start getting the main loop of commands and movement working.

[Dean Anderson]

Well, I've converted the first couple of thousand lines of code so far, and it's not going too badly.

When it comes to a port of this size, what to do first is a very interesting question. Do you start with some kind of Minimum Viable Product where you have a character walking around and then start adding features? Or do you take a more monolithic approach and start at the "beginning" of the code that runs when you start the program and go from there?

The former gets you quicker visible results, and is almost certainly better if you're writing a game from scratch, but it can have the problem that it's less methodical and you end up missing things in a port, so I've gone for the latter approach.

I obviously started with the "Get an 80x45 grid of ascii characters to display and update, with print functions" basics, but after that I've gone for starting at the beginning.

I mentioned in the first post that I'd got as far as character creation. It now does post-character-creation initialisation too, loading all the template data for monster type and item types and so forth, initialising the flavours for potions and scrolls and things, and creating the wilderness map, towns, and dungeons.

Next on my list is creating the town level, and then we'll eventually have got to the point where we can do the basic turn structure and display code to let you walk around...

From a language point of view, it's not too bad. GDScript is a proper OOP language with classes and inheritance, so I need to do a lot less restructuring than I did when moving from C to C# where I had to convert from procedural code to class-based code. I can mostly simply port functions one by one.

Sure, the syntax is different, going from a C-based language to a Python-based language, but for the most part (at least so far) that's something that requires small-scale conversions on a line by line basis. I can keep the same classes with the same methods and properties and just convert the individual implementations.

The main sticking points are that GDScript doesn't have interfaces and the way input is handled.

Input is a particularly sticky issue.

In the WPF version you had an "Application.DoEvents()" command which would tell the application to handle all pending events from the OS. This includes things like the window moving or being minimised, but also keypresses. This meant that I could have a simple "GetKey" fumction which was basically an infinite loop that checks the key buffer for a key with a DoEvents inside it. The loop starts running, and if the user presses a key it an event is fired which adds the press to the key buffer. The function then detects this and breaks out of the loop returning the key.

Unfortunately for me, there's nothing similar in Godot. Instead it follows the standard game-engine paradigm of having a render loop and a process loop. There's no simple way to pause mid-code in a function and wait for a keypress before continuing, because while you're waiting the engine isn't processing events. Instead you have to do one of two things.

The first is to make a state machine where instead of a single process the game runs in little single-input chunks and then each chunk finishes and it goes back to idling before a keypress happens and it runs again to process that input. That's fine for some things, but it falls apart when you have to do things like ask the user for something in the middle of a complex command. There's just too much state information to persist between inputs.

The second is to use the "yield" paradigm for co-routines (although Godot implements this with an "await" command). So you can have your process running code and then use "await" to wait for a specific event such as a keypress event before carrying on. This sounds like what the WPF was doing but the important difference is that it doesn't pause the process/thread. Instead it drops out of the function and continues processing, returning to complete the function later when it gets the event.

Naturally, this isn't what we want, because we want to stop and wait for a keypress rather than continue on with whatever function called the GetKey function. You can get around it by having the outer function await the completion of the GetKey function, but this soon results in proliferation of awaits and ties the whole game up in a mess.

So I'm using a hybrid of the two. I've got a signal bus that sits there receiving keyboard inputs and then has a bunch of events that it can fire. It checks the state we are in and then fires the relevant event, and the various game components (menu, load-screen, game-screen) all catch only their specific variety of event. This means that most of them can idle between function calls and just wake up when the game is in the correct state.

However, for the more minor places where input needs waiting for, like asking for quantities or targets mid-command or something, I use await.

This means that I don't need special stats for the minor places and it means I don't need to proliferate the awaits all the way through the code, only down to the "process the player's in-game command" function.

[Nick]

You're braver than me

[Dean Anderson]

Some of you may remember that a few years ago (2018, so that's eight years ago now) I ported Cthangband from C to C#.

While this allowed me to remove decades of cruft from the code and make it a much cleaner object-oriented codebase, it had a couple of disadvantages. Firstly, I was compiling it as a WPF application which restricted it to Windows only, and secondly I was relying on the serialization capabilities of the .NET Framework 4.5.

Time makes fools of us all, and now both of those have come back to bite me.

Around a year ago, I switched from Windows to Linux due to Microsoft's increasing enshittification of Windows 11, Unfortunately this means that without breaking out a virtual machine I can't even run Cthangband any more, let alone compile it. And if I did want to compile a new version, I'm stuck with .NET 8 which doesn't have the old serialization functionality (it was removed for security reasons).

So, like the masochist I am, I'm porting it again...

I've been using Godot game engine for the last year, making a conversion of Ularn as a practice project, and I'm now reasonably confident with the platform. It has the advantage that it can build Linux, Mac, Windows, and HTML 5 Web versions all natively so it means I get cross-platform compatibility back, and it's also FOSS, which is a bonus.

It also has the advantage that I will be able to put the source for the Godot project online and people will be able to dowload/compile it themselves rather than needing to trust the download of a precompiled executable or installation script.

The big disadvantage is that Godot's native language - GDScript - is Python-based rather than C-based. So once again I'm having to port every line of code (a quick check tells me there are around 125,000 lines of code in Cthangband) from one language to another with a different syntax.

Except that this time I'm also having to switch from procedural code that occasionally idles waiting for a keypess to a fully event-driven model that runs in a main render loop and fires out events (or "signals" in Godot-speak) when keypresses and other input happens and which expects most of your code to run between frames.

I've not got very far yet, but the basic structure and user-interface is there and I've got as far as handling the main menu, loading/creating a new game, and going through the character creation screens. It's surprising how much I've needed to implement to get that far, because even simple chargen things need a lot of data and other systems to be set up.

For example, to be able to display the character sheet at the end of the chargen process I need to call the function that updates your character when a change has happened (in this case, the change being that the character is newly created). This function takes into account what your character's ancestry and profession are, which is fine because the basics of those needed to be implemented anyway for you to be able to select them), but also what effect mutations have on your abilities, so I had to implement the mutation system, and what effect your items have on your abilities, so I had to implement parts of the item system, and when your character's birthday is and whether they start at night or during the day, so I had to implement the game's timekeeping system.

So in terms of implementation, I'm taking a depth-first approach, slowly moving forwards in terms of progression into the game while diving deep into the subsystems required as I take each step forwards.

It's going to be a long haul, though.