Pyrel dev log, part 4

[Magnate]

Just sayin' ;-)

[Derakon]

Yeah, yeah. Wimp.

Anyway, fizzix expressed some confusion about how the code is laid out, which is entirely reasonable for someone starting on a new project, so I went ahead and wrote up a high-level overview of the code. It's not very well-edited (but it's a wiki! Hint Hint!), but it should help people get oriented.

Also, an idea I had that I need to record somewhere: use the same system for displaying the character's status as for displaying monster memory. Obviously the character has some stats that monsters don't (currently) care about, like perception, stealth, and device use; monsters also have different spells. But the overall framework is similar, and I think it's a good idea to reinforce how Angband's systems work especially e.g. with regard to resistances.

The by-equipment breakdown page would be a follow-on page, of course.

Incidentally, it would totally be possible to write up monster memory now; monster stats are fully fleshed-out.

[Derakon]

Not to worry, things aren't dead! In fact, Pyrel has seen a lot of activity lately. Magnate's gotten a lot done on item affixes; Fizzix is laying the groundwork for level generation; Noz has been adding various wizard-mode commands. And for my part, I've mostly been approving pull requests.

The next thing I personally want to tackle is a refactoring of the proc/effect system. The original system has been extended and over-extended and is now more painful to work with than it really needs to be. Part of the problem is that every Proc is bundled into an Effect, which creates a lot of excess layers in the data files. For example, if you want to make a potion that cures 10% HP when drunk, you'd:

* Create a record in object.txt for the potion
* Create an Effect for the potion that triggers on item use and invokes the "healPlayer" Proc
* In the potion's record in object.txt, set the healing amount for the proc to 10%

This is a lot of bouncing back and forth between object records and effect records, and the code is kind of messy too. So here's my proposal for a new system:

* The datafiles get a new file: data/proc_template.txt. There is no data/proc.txt (but entries in proc_template can be "complete" procs if desired).
* Each individual proc has a name (referring to the code it will invoke), a trigger condition (determining when its code will be invoked), and any parameters it requires. Of course, some of these may be filled in by templates. However, the trigger condition is not specified either by the proc record or by its templates.
* When procs are referred to in other records, it is as a mapping of trigger conditions to lists of procs.

I think this covers the use cases we want. The important thing is that we want to be able to use the same in-game "effect" from multiple different triggers, without having to create separate proc records for each trigger. Under this regime, we'd have something like the follow:

proc_template.txt:
{
    "name": "modify hitpoints"
}

object.txt:
{
    "name": "Cure Light Wounds",
    "templates": ["potion"],
    "procs": {
        "on item use": [
            {"name": "modify hitpoints",
              "target": "user", 
              "power": "+10%"
            }, 
            "decrement quantity"
        ]
    }
}

Of course, we do have a lot of potions that heal the user by some amount, so we might want to have a separate "heal self" proc that in-lines the "target" parameter. And we could modify the "potion" object template so that "on item use" always invokes the "decrement quantity" proc. But the basic concept seems sound.

Thoughts?

[Derakon]

Therem Harth expressed some interest in working on spellcasting for Pyrel, so I guess I'd better write down my thoughts on that!

We can think of spells as having three main components: the effect of the spell when cast, the cost of casting the spell, and the chance of failing to successfully cast it. In Pyrel, each of these should be a Proc. How are these Procs handled? We can attach them to items -- call them spellscrolls.

And the items, of course, go into spellbooks, which in Pyrel will be containers. Of course in Vanilla proper you would be unable to put things into or take things out of spellbooks, since they'd be generated with all of their spells already set, but that's a detail that can be saved for later. In fact, spellbooks as a whole can be saved for later -- just implementing working spellscrolls will be enough to get us started.

So the player's holding some spellscrolls and they choose the "cast a spell" command. This command needs to do the following:

Gather this information:
* Which spell the player wants to cast (ItemListPrompt to select a spellscroll)
* How much it costs (examine the selected spellscroll)
* Whether the player really wants to cast it (YesNoPrompt), if the cost exceeds their current allowance
* Any relevant targeting information (TargetPrompt or ItemListPrompt as appropriate)

And then execute:
* Deduct the cost from the player's stats
* Determine if the player succeeds in casting the spell
* Cause the spell's effect to take place


Now we need to decide what the item record looks like. I suggest something like the following:

{
    "subtype": "&Resist Heat & Cold~",
    "templates": ["spellscroll"],
    "allocations": [{"commonness": 20, "minDepth": 10, "maxDepth": 60}]
    "procs": [{
        "triggerCondition": "cast spell", 
        "name": "temporary stat mod", "modAmount": 1, 
        "statNames": ["RES_FIRE", "RES_COLD"], 
        "duration": "10+d10M20",
        "targeting": "none"
    },
    {
        "triggerCondition": "spell failure rate",
        "name": "failure rate calculator", 
        "difficulty": 5,
        "spellType": "holy"
    },
    {
        "triggerCondition": "spell cost",
        "name": "stat calculator",
        "statName": "holy skill",
        "multiplier": -.5,
        "addend": 10
    }]

Let me explain some of this now. First off, the "spell cost" proc here is invoking an as-yet nonexistent proc that calculates a number based on the player's stats. Here we're saying that the spell's cost is 10 - half the player's holy skill stat. So we can go into the Priest class template and give them a holy skill of 10, say (spell cost is 5), and then the Paladin gets a holy skill of 5, so they pay 2.5 extra mana to cast the spell (spell cost is 7).

Then the spell failure rate proc is a specific proc only for dealing with spells. It too operates on the player's stats, and all it cares about is how hard the spell is to cast. Presumably it uses a formula such that the higher your skill and the higher your spellcasting stat, the easier the spell is to cast.

Finally the "cast spell" proc actually causes the spell to take effect, by giving the player a temporary bonus to their fire and cold resistance stats. These stats are actually binary on/off but codewise we treat them as if they stack. Actually we should use a separate stat name for temporary and permanent element resistances so that we can use Vanilla's..."unique"...approach to resistance stacking, but that's an unimportant detail.

Note that the "cast spell" proc has an extra "targeting" field. The proc code itself will not refer to this, but the command-execution logic needs to know how the spell is targeted so it can determine what information is needed to execute the spell. Valid targeting values would be "none", "item", and "aimed" -- that last one is for normal projectile targeting. Of course we need to modify the Proc code to provide access to this information; just add this function to the base Proc class:

def getParam(self, paramName):
    if paramName not in self.params:
        raise RuntimeError("Invalid parameter name [%s]" % paramName)
    return self.params[paramName]

Anyway, that should about do it. Thoughts? Requests for clarification? Thread-derailing tangents?

[fizzix]

Let me explain some of this now. First off, the "spell cost" proc here is invoking an as-yet nonexistent proc that calculates a number based on the player's stats. Here we're saying that the spell's cost is 10 - half the player's holy skill stat. So we can go into the Priest class template and give them a holy skill of 10, say (spell cost is 5), and then the Paladin gets a holy skill of 5, so they pay 2.5 extra mana to cast the spell (spell cost is 7).

I don't like this. You want some classes to be better at certain spells than other classes independent of a holy or magic skill stat. For example, a rogue should be better at detecting magical objects and a mage, but a mage is better at creating magical light.

In other words, make sure we can support the V design where each class has a list of what spells it can cast and what they cost.

[Patashu]

What if I wanted, say, an amulet that could be used to cast a spell? Would the code that prompts me to choose a spell to cast also iterate over non spellscrolls to see if they have the trigger 'cast spell'?

If I wanted a spell that could only cast in certain situations, would I give it a novel proc under spell failure rate that makes the failure rate 100% when you're not allowed to cast it?
Would it be possible to combine procs that resolve to a numerical or boolean type via min/max/and/or/xor?
It would also be useful to have a generic expression syntax to sneak into places where a calculation is expected (for example, if modify hit points took current hp as x and max hp as y and could be set to 020, 2d12, y*0.1, -x*0.9 for poison, max(2d12+y*0.1), etc). Being able to do such a thing could save on lots of dummy procs that do the same as other procs but with a brief calculation first. I've written an expression evaluating calculator in C# once, took 2 days

[Derakon]

I don't like this. You want some classes to be better at certain spells than other classes independent of a holy or magic skill stat. For example, a rogue should be better at detecting magical objects and a mage, but a mage is better at creating magical light.

In other words, make sure we can support the V design where each class has a list of what spells it can cast and what they cost.

You have a fair point that some classes ought to be better at certain types of spells than other classes, but IMO we shouldn't do this by just enumerating the spells and classes and declaring a skill level for each one. Much better to come up with a set of rules that describe the system that we want, so that later when we add new spells we just insert them into the system and they automatically behave themselves nicely.

Thus what I'd suggest is to split the skill stats:
* healing: priests > paladins > rangers >> mages/rogues
* detection: rogues > mages > priests > paladins/rangers
* attack: mages > priests >> rangers > paladins >> rogues
* buffing: priests > mages > paladins? > rogues? > rangers
etc...

To make spells exclusive to certain classes, we can have a failure-rate proc that flat-out prevents casting the spell if your skill with that spell type is below some threshold level. Spells with 100% failure rates are unlearnable.

This leads into Patashu's question...

What if I wanted, say, an amulet that could be used to cast a spell? Would the code that prompts me to choose a spell to cast also iterate over non spellscrolls to see if they have the trigger 'cast spell'?

Yes. When the item is created, it examines its procs; if it has procs with the "cast spell" triggerCondition then it puts itself into the CASTABLE container, which the "cast a spell" command will use to select possible spells to cast.

Put another way, any item that has a "cast spell" proc can be used to cast a spell.

If you have no "failure rate" procs, then the spell always succeeds; if you have no "spell cost" procs, then the spell is free to cast.

If I wanted a spell that could only cast in certain situations, would I give it a novel proc under spell failure rate that makes the failure rate 100% when you're not allowed to cast it?

I didn't get into this in the original post because I didn't want to get bogged down, but my assumption is that if you have multiple procs with the same trigger condition, then they are attempted in sequence. So if you have multiple failure-rate procs, then if any of them fails the spell fails. Likewise you could have multiple spell-cost procs (say, to make a spell temporarily drain a stat in addition to costing mana) and they would all apply when the spell is cast. And of course the spell could have multiple effects when it is successfully cast.

It would also be useful to have a generic expression syntax to sneak into places where a calculation is expected (for example, if modify hit points took current hp as x and max hp as y and could be set to 020, 2d12, y*0.1, -x*0.9 for poison, max(2d12+y*0.1), etc). Being able to do such a thing could save on lots of dummy procs that do the same as other procs but with a brief calculation first. I've written an expression evaluating calculator in C# once, took 2 days

I spoke with Magnate about creating a Proc that does multivariable polynomial evaluation without having to do any parsing -- the expression would be provided as a list of coefficients (e.g. "[60, 3, .5]" => "60 + 3x + .5x^2"). This is a bit clunky but does work. If we wanted to have proper expression parsing, and that might well be useful for making the data files easier to read if nothing else, then we can either try to implement our own in-house parser, or use a third-party library, of which there are plenty.

The only problem with third-party libraries is that they create extra dependencies that people have to install before they can start developing the game. Players wouldn't care as the game would be distributed as a standalone executable with all dependencies bundled in, but ideally devs should be able to get up and running with a minimum of fuss.

So hey, if you feel like rewriting your expression evaluator in Python, then I for one wouldn't say no.

[fizzix]

You have a fair point that some classes ought to be better at certain types of spells than other classes, but IMO we shouldn't do this by just enumerating the spells and classes and declaring a skill level for each one. Much better to come up with a set of rules that describe the system that we want, so that later when we add new spells we just insert them into the system and they automatically behave themselves nicely.

Thus what I'd suggest is to split the skill stats:
* healing: priests > paladins > rangers >> mages/rogues
* detection: rogues > mages > priests > paladins/rangers
* attack: mages > priests >> rangers > paladins >> rogues
* buffing: priests > mages > paladins? > rogues? > rangers
etc...

To make spells exclusive to certain classes, we can have a failure-rate proc that flat-out prevents casting the spell if your skill with that spell type is below some threshold level. Spells with 100% failure rates are unlearnable.

I disagree pretty strongly with this. The number of different spell categories that most devs will want is going to be so large as to completely neuter its purpose and add an extra layer of complexity instead of providing simplification.

What we should be aiming for is that the creation of new spells (or items, monsters, level types etc) is simple to understand, not that it shouldn't require some thought or effort in the values. Requiring a person creating a new spell to specify what classes can cast it, what it costs for them, and what the failure rate is for each of them seems like a perfectly fair requirement and not overly burdensome in any way. Actually, to me it seems far easier than figuring out some complicated weighting scheme for different classes.

The only question to me is where the information goes. It could go in the spell or the class. It seems in the pyrel way, it should all go in the spell information, so each spell would have a list of class (or race?) flags that can cast it, what it costs, failure rate, and any other spell characteristics (like xp gain for first casting).

[Derakon]

I disagree pretty strongly with this. The number of different spell categories that most devs will want is going to be so large as to completely neuter its purpose and add an extra layer of complexity instead of providing simplification.

I don't personally think this statement can be supported, but frankly it doesn't matter because we aren't locked into a single system. If you want to have spells that use hardcoded variations depending on the player's class, then you can do that. Just make a different set of procs for determining failure rate / spell cost, and, yes, include the class-specific values in the procs' definitions.

I daresay most spells can simply use the skill-based system instead and only a few exceptions, if any, will need the more specific approach.

[Patashu]

So hey, if you feel like rewriting your expression evaluator in Python, then I for one wouldn't say no.

Ok, I copied how I did it from memory.

(REDACTED)

Probably still buggy, haven't had much chance to test yet. Will make sure everything works next time I sit down at a computer

Using python's eval() may be a possible alternative, btw.

[Derakon]

Using python's eval() may be a possible alternative, btw.

No, it isn't. Anything that treats data as code is a gaping security hole and has no place in a serious project. Which is unfortunate because it makes things very easy.

[AnonymousHero]

Ok, I copied how I did it from memory.

http://pastebin.com/GPTJVsvR

Given your evaluator, what is the result of "(8+5)*3"? Or "5+abs(5*abs(-5))"?

Your code looks hideously complicated since you're (it seems to me) combining parsing and evaluation. Those concerns should be separate -- if you separate them you'll get completely trivial code which is "obviosuly" correct.

EDIT: I also imagine that you'll want to be able to calculate different min, max and average(?) based on the expression for display purposes and that'll be easier to do cleanly if you've separated parsing from evaluation.

Probably still buggy, haven't had much chance to test yet. Will make sure everything works next time I sit down at a computer

You should add a "nose" test suite containing various simple and comlex evaluations along with their expected results. That way you can document that the code works and help *keep* it working.

[Patashu]

Sit down at home, review Calculator.cs for C#, hammer out bug fixes for 2 hours \o/ (Most mistakes were from not being used to python, a few were forgetting important things like the lookaheads/lookbehinds on unary operators so 2V2 doesn't turn into 21.414etc, and getting order of operations right for operators of the same precedence)

EDIT: http://pastebin.com/H07vMSEf fixed 4-5*-5 and similar expressions

Everything not-completely-crazy should work now

Given your evaluator, what is the result of "(8+5)*3"? Or "5+abs(5*abs(-5))"?

>>> calculatethis("(8+5)*3")
39.0
>>> calculatethis("5+abs(5*abs(-5))")
30.0
>>>

[Magnate]

Sit down at home, review Calculator.cs for C#, hammer out bug fixes for 2 hours \o/ (Most mistakes were from not being used to python, a few were forgetting important things like the lookaheads/lookbehinds on unary operators so 2V2 doesn't turn into 21.414etc, and getting order of operations right for operators of the same precedence)

http://pastebin.com/20vCSF7Y

Everything not-completely-crazy should work now


>>> calculatethis("(8+5)*3")
39.0
>>> calculatethis("5+abs(5*abs(-5))")
30.0
>>>

Thanks Patashu - that is just fantastic, and perfect timing for me. After my discussion with Derakon I had moved to the point where I was just about to suggest using a 3rd-party library, but now we can try to avoid that.

[Magnate]

I don't personally think this statement can be supported, but frankly it doesn't matter because we aren't locked into a single system. If you want to have spells that use hardcoded variations depending on the player's class, then you can do that. Just make a different set of procs for determining failure rate / spell cost, and, yes, include the class-specific values in the procs' definitions.

This is just brilliant.

I daresay most spells can simply use the skill-based system instead and only a few exceptions, if any, will need the more specific approach.

I think I'd go so far as to say that our aim should be that the skill-based system should replicate V costs and expertise in the first instance. Variants can then use whichever system they like.

I just realised how easy it will be to reimplement Sangband in Pyrel ....