Pyrel dev log, part 3

[Derakon]

I've thrown together a to-do list based on my previous post, and put it on the wiki. It's pretty rough but it should give some idea of what important stuff still remains to be done.

Uh...not much else besides that. I think my priorities for the moment should be on enabling as many effects and procs as possible -- porting in all of the effects from Angband itself is a huge task, but fortunately one that can be broken down into hundreds of smaller tasks. In other words, it's an ideal way for people to get their feet wet with Pyrel development.

[ekolis]

Regarding character generation, there's no reason to have to create a new character every time you start the game just because there's no way to save the game - you could implement saving just for character objects first!

[Derakon]

Any amount of save/load implementation has to have a fairly significant framework built up first if it's to be done properly. Kind of like how you can't implement spells without having a lot of the effects framework done first -- you can't say "Oh, just implement the spells and leave the rest for later" because you aren't actually saving yourself all that much work.

EDIT: Ekolis's pretty-item-names patch is finally in! And I've also uploaded the first pass at animations. You can throw any item in your inventory to see an animation of it traveling -- it won't actually be thrown, but seeing as combat is basically nonexistent right now I consider that to be a relatively minor problem.

[ekolis]

Well, you could just hack something together with pickle and not worry about versioning...

[AnonymousHero]

You should seriously consider adding automated testing at this point -- I can recommend mock-based testing as that tends to be easiest for developing + testing small bits in isolation. Retrofitting automated tests after all the code is written can be a seriously large undertaking... which means that people don't tend to do it... which means that (esp. in a dynamically type checked language) you'll start to instability start to creep in, getting worse and worse over time.

Having a "standard" way to do tests would probably help contributors out a lot too.

[Derakon]

I'll be honest, writing tests is not my favorite thing to do in the world, so I'm less than enthusiastic about this. At the same time, I do recognize that tests have value, and you're right, it's easier to get them in sooner rather than later.

That said, they should probably wait until save/load is implemented at least; it'll be far easier to test different scenarios if you can just load an associated savefile that does your setup, rather than having to manually construct all the different objects you want at the start of the test.

So I guess save/load is the next bit I get to tackle.

[Magnate]

I'll be honest, writing tests is not my favorite thing to do in the world, so I'm less than enthusiastic about this. At the same time, I do recognize that tests have value, and you're right, it's easier to get them in sooner rather than later.

That said, they should probably wait until save/load is implemented at least; it'll be far easier to test different scenarios if you can just load an associated savefile that does your setup, rather than having to manually construct all the different objects you want at the start of the test.

So I guess save/load is the next bit I get to tackle.

I'd be willing to tackle tests, if I can get sufficiently up to speed with Python. Will contact you on IRC.

[AnonymousHero]

I'll be honest, writing tests is not my favorite thing to do in the world, so I'm less than enthusiastic about this. At the same time, I do recognize that tests have value, and you're right, it's easier to get them in sooner rather than later.

That said, they should probably wait until save/load is implemented at least; it'll be far easier to test different scenarios if you can just load an associated savefile that does your setup, rather than having to manually construct all the different objects you want at the start of the test.

So I guess save/load is the next bit I get to tackle.

Actually, with mock-based testing you can avoid this. Instead of having some sort of large corpus of specially defined loadable test-data, you instead (in each individual test) provide mocks for all dependencies (including data) of the functionality you're testing. (It's kind of hard to explain, the wiki page does a better job!)

The great thing about mock-based testing is that you can tame the combinatorial explosion that often occurs when you're testing full systems using "real" data.

[Derakon]

As you may have noticed, Pyrel development's slowed down these last couple of weeks. Work suddenly gave me some interesting new problems to tackle, and they've been sucking up all of my spare "programmer time" -- it's not that I'm working longer hours, it's that I don't have the brain to devote to Pyrel when I get home. Hell, I tried to play board games last Friday and frankly was an embarrassment.

Still, I wasted away the weekend playing FTL when I could have been getting useful things done, so there's some missed opportunities there.

Anyway, I need to set my subconscious loose on some problems at work, so let's talk about where Pyrel should go next.

What I think really needs to be worked on isn't so much saving and loading as it is a fully-fleshed out player character. That would be a big enabler for all sorts of subsystems, which would in turn help unblock Pyrel development from requiring my personal involvement. For example, combat really wants a character with STR and DEX stats, while we want a Magic Devices skill for wands/rods/staves/etc. Traps really want to be able to drain stats. And so on. My concern earlier about having to deal with chargen at the start of every play session can be readily worked around by simply tossing the player into a hardcoded race/class combination -- nobody using Pyrel at this stage should be incapable of changing the combination if they need to.

So what defines a character, then? To over-reduce, characters are collections of summations of numbers. For example, the character's magic device skill is a summation of an INT-based modifier (itself being an "internal" stat plus race, class, and equipment bonuses) plus a class-based multiplier times the character's level, plus a racial modifier. Very few of these values are what I would consider to be innate:

* The aforementioned internal stats.
* Current experience total.
* The base d10 hit die.

Everything else, including racial and class-based bonuses, I think are sufficiently subject to change that they should be treated as modifiers on these base stats (or modifiers on 0 when there's no equivalent base stat, e.g. for skill checks).

So how do we represent this internally? By a list of values, of course. Except that those values may be represented by functions (e.g. multiplying level by the skill gain rate), and might become even more complicated in future. So let's abstract things fully by creating a new class, the StatMod. Each stat is represented by a list of StatMods; the sums of their evaluate() methods provides the actual value to use. We gain several things by this:

* Flexibility. We can replace just about anything about the character with something else and the system will seamlessly adapt. Want to change race? Done. Class? Done.
* Effects can affect stats in cleanly-separable ways. An Effect simply inserts a new StatMod into the sequence while it is active, and removes that StatMod when it is done.
* As a net result, handling equipment bonuses isn't a huge nightmare of special-coding. Equipment bonuses have an onEquip proc to add a StatMod and an onUnequip proc to remove it; done and done.

There are however two caveats to this approach:

* Every time we want to get the value for a stat, we must recalculate it. This cost should be minimal most of the time, but it does gall a bit. Values could be cached, but then I imagine a system where e.g. the value of your magic device skill depends on your proximity to a pylon that's sending out magic-disrupting energy waves. Figuring out when the cache is invalid could be aggravating.

* There's potential for circular dependencies here. For example, an amulet that increases your INT based on your magic device skill seems sensible, but your magic device skill is derived from your INT -- so when you go to calculate it, you have a loop that can never complete. The simple answer is "don't do that then"; more complicated would be to have some way to short-circuit calculations when a loop is detected. But that's significantly more complicated -- we're talking about metaprogramming here, assuming that we want to apply bonuses in a fixed order. And if we don't...

* Percentage increases to the stat have a similar problem. If you have an item that gives you a 10% bonus to your stat, when is that 10% applied? Given that e.g. there's no "base" magic device stat (it starts from 0 every time), you can't just apply it to the base. Are we to prioritize the order in which modifiers are applied?

On a side note, similar logic to the above can and should be applied to items. Currently items have two sets of modifiers: an "innate" set (as applied in the item's own record) and an "effective" set (including bonuses from external factors like Rings of Prowess), and they just assume that everyone will do the math correctly. But if we're going to implement this more flexible StatMod approach for the character, and we should, then doing the same for items seems like a no-brainer.

[fizzix]

* There's potential for circular dependencies here. For example, an amulet that increases your INT based on your magic device skill seems sensible, but your magic device skill is derived from your INT -- so when you go to calculate it, you have a loop that can never complete. The simple answer is "don't do that then"; more complicated would be to have some way to short-circuit calculations when a loop is detected. But that's significantly more complicated -- we're talking about metaprogramming here, assuming that we want to apply bonuses in a fixed order. And if we don't...

I think a hierarchical dependency tree makes sense. Device skill can depend on INT, but INT cannot depend on device skill. The other option is to hard code a way return if the bonus is less than one. Then provided the bonuses decrease with each call you will exit the loop eventually. I much prefer the first approach.

Basically you have the very top level stuff which are the stats. They get derived from XP, racial bonus, class bonus, equipment, effects. On the second level you have the derived stats. These are stuff like prowess, finesse (or just melee), stealth, magic device skill etc. These can depend on all the stuff that the stats depend on, but also on stats. However, stats cannot depend on each other or on any of the derived stats.

I think this also makes intuitive sense.

* Percentage increases to the stat have a similar problem. If you have an item that gives you a 10% bonus to your stat, when is that 10% applied? Given that e.g. there's no "base" magic device stat (it starts from 0 every time), you can't just apply it to the base. Are we to prioritize the order in which modifiers are applied?

This one is much harder. In order for percentages to make sense you need a base to calculate off of. The current approach would rule out percentage based stat modifiers. But maybe that's a sacrifice worth making?

[Derakon]

I think a hierarchical dependency tree makes sense. Device skill can depend on INT, but INT cannot depend on device skill. The other option is to hard code a way return if the bonus is less than one. Then provided the bonuses decrease with each call you will exit the loop eventually. I much prefer the first approach.

Basically you have the very top level stuff which are the stats. They get derived from XP, racial bonus, class bonus, equipment, effects. On the second level you have the derived stats. These are stuff like prowess, finesse (or just melee), stealth, magic device skill etc. These can depend on all the stuff that the stats depend on, but also on stats. However, stats cannot depend on each other or on any of the derived stats.

I have to say I don't much like the idea of having a hardcoded hierarchy of stats and saying "this stat cannot depend on this other stat", if only because it assumes an awful lot about what is more "integral" than what.

However, it occurs to me that there's a natural hierarchy to the modifiers of stats:

Base > innate > derived > permanent effect > temporary effect

In other words, a base stat ("base modifier") is clearly more fundamental than a modifier that is derived from base stats, while a permanent effect (e.g. from equipping a Ring of Strength) is more fundamental than a temporary one. So what we can do then is to ascribe a tier to each kind of modifier, and only allow modifiers of a strictly lesser tier to be used when calculating that modifier.

For example, say we have a weapon with a modifier that gives a 20% bonus to STR. The character has a base 12 STR (internal score), an innate +3 to that bonus (half-orc or something), and has no derived bonus. The weapon's modifier counts as a "permanent effect", so nothing besides these values can be considered. Thus the weapon gives +3 STR (20% of 15).

If the character had another item that also gave a 20% bonus to STR, then that item would also give a +3 STR bonus, since none of the values it cares about has been affected by this weapon.

I believe this approach should eliminate the possibility of loops while still allowing a good deal of flexibility. The only tricky bit is determining what tier a modifier should get. The above hierarchy works well for determining player stats, but we'll need a subtly different system for determining mods on items (since we need to be able to differentiate between the modifiers on the item itself, which count as "innate" tier, from external permanent modifiers, which count as "permanent" tier). Still I think that ought to fall out without too much difficulty.

[Magnate]

I have to say I don't much like the idea of having a hardcoded hierarchy of stats and saying "this stat cannot depend on this other stat", if only because it assumes an awful lot about what is more "integral" than what.

However, it occurs to me that there's a natural hierarchy to the modifiers of stats:

Base > innate > derived > permanent effect > temporary effect

In other words, a base stat ("base modifier") is clearly more fundamental than a modifier that is derived from base stats, while a permanent effect (e.g. from equipping a Ring of Strength) is more fundamental than a temporary one. So what we can do then is to ascribe a tier to each kind of modifier, and only allow modifiers of a strictly lesser tier to be used when calculating that modifier.

For example, say we have a weapon with a modifier that gives a 20% bonus to STR. The character has a base 12 STR (internal score), an innate +3 to that bonus (half-orc or something), and has no derived bonus. The weapon's modifier counts as a "permanent effect", so nothing besides these values can be considered. Thus the weapon gives +3 STR (20% of 15).

If the character had another item that also gave a 20% bonus to STR, then that item would also give a +3 STR bonus, since none of the values it cares about has been affected by this weapon.

I believe this approach should eliminate the possibility of loops while still allowing a good deal of flexibility. The only tricky bit is determining what tier a modifier should get. The above hierarchy works well for determining player stats, but we'll need a subtly different system for determining mods on items (since we need to be able to differentiate between the modifiers on the item itself, which count as "innate" tier, from external permanent modifiers, which count as "permanent" tier). Still I think that ought to fall out without too much difficulty.

This looks fine with me - I don't think it will be too hard to differentiate innate from permanent. I think this solves both the percentage problem and the circular dependency problem. I don't think recalculating stats is going to be at all problematic - the only time anyone will notice is when the stats generator starts doing stats on combat, simulating thousands of combats ... and that's quite a long way away.

So, looking good - once I can get noz to explain OO coding to me, I'll get cracking!

[AnonymousHero]

So what defines a character, then? To over-reduce, characters are collections of summations of numbers.

Slightly more generally a character is a monoid formed by a binary associative operator on sets of modifiers (+stealth, combat bonuses, +INT, -DEX, XP drain, etc.). Everything follows from that.

In my theorycrafting I haven't found anything as of yet which is 1) meaningful as a character mechanic, and 2) cannot be accomodated in the above general model.

[Derakon]

That was surprisingly painless. I suppose it helps that only two stats were currently being used by the engine (max HP and speed). All Creatures now have a "stats" field which contains all of their mutable stats except for current HP (I thought about it and couldn't come up with a scenario in which temporary modifiers to current HP couldn't be accomplished some better way).

Note that this doesn't mean I've added support for any player stats; they're still just the monster standards. Stuff like awareness, absorption, evasion, etc. I'm arbitrarily deciding that native depth, rarity, and experience reward are "fundamental" and it doesn't make sense to stick them into the Stats container. I'm open to arguments though. I mean, if you wanted to make summoned monsters be worth 0 experience, then you could stick a permanent StatMod on them that multiplied their stat reward by 0, but again, that's not the only way you could do that...

I pruned the list of stat types down to "inherent", "permanent", and "temporary". I was having trouble distinguishing between fundamental and inherent stats anyway, and figuring out how the program would decide where a fundamental stat is defined vs. where its inherent modifiers should up was simply not worth the effort.

I also implemented Potions of Speed, and they work quite nicely. You can implement any temporary stat modifier you like without having to touch the code at all since there's now a generic "apply temporary stat mod" proc you can use.

Implementing this proc required me to implement some kind of timer, which in turn meant making a new updateable Thing (since the timer passes some number of normal-speed turns), which lead me to realize that I wanted to change how updateable Things are handled. Currently any Thing that wants to be updateable has to manually provide several attributes -- it must have a stats object that has a 'speed' stat; it must have an energy value; it must have a name (for sorting when two entities have the same speed); it must have a getStat() method (to retrieve the speed stat)...all of this is practically irrelevant to most entities; they just want their update() methods to be called.

Now, I could avoid having to re-do this by having "Updateable" be a descendant of Thing, which both Creature and Timer would then descend from in turn. But you start running into problems pretty quickly there if you have several different kinds of capabilities that you want Things to have -- the inheritance trees would get ridiculous pretty quickly, and multiple inheritance is generally a Bad Thing. So instead, you add the updateable "capability" to an entity via mixins. You can think of mixins as inheriting just the functionality you want -- a mixin is an "incomplete class" that can't stand on its own.

So in this case, the Updateable class has update() and addEnergy() methods, and its constructor ensures that the instance has the appropriate fields to use those methods properly, but it doesn't do anything else.

Later on, I may decide that e.g. the ability to carry, equip, and drop items should be a mixin, in which case the "inventory", "equipment", pickupItem(), dropItem(), etc. attributes of the Creature class will be moved out to a new mixin class. This could be useful for e.g. containers (as Item Things that can "pick up" and "drop" other items) or for socketable items (Items that can "equip" other items).

Anyway, this all is on the repo now. Hooray progress!

[ekolis]

I mean, if you wanted to make summoned monsters be worth 0 experience, then you could stick a permanent StatMod on them that multiplied their stat reward by 0, but again, that's not the only way you could do that...

What kind of operations can a StatMod do? Just addition and multiplication? Or is it more of an "arbitrary function" sort of thing with an input and an output?

it must have a name (for sorting when two entities have the same speed)

So if you have two entities with the same speed, the one that's first alphabetically gets to go first? That just seems... wrong... Maybe just pick one at random instead?

You can think of mixins as inheriting just the functionality you want -- a mixin is an "incomplete class" that can't stand on its own.

I've heard of mixins before - I thought they were more along the lines of mixing code in several different languages to accomplish a task, though? What you describe sounds more like interfaces in C# and Java, except that (perhaps) you can combine two or more mixins to get the full functionality of a class, without actually having to inherit from said class in whatever is implementing the mixins...