Pyrel dev log, part 4

[Mikko Lehtinen]

Skill-based magic sounds really cool.

In Halls of Mist I'm going to have about the simplest possible skill-based spellcasting system. Learn spells by reading books. True spellcasting classes gain random bonus spells (from either Arcane or Divine list) on certain character levels. To cast any spell you have learned, roll 1d100 <= Spellcasting skill. All spells have a "mastery" level. When you reach that character level, you get to reroll a failed check once.

The exact same system is used for Magic Devices.

[fizzix]

I will reserve my objections, but I will not pursue it further until I have a chance to see how everything works in actuality. Sounds fair?

[Derakon]

I will reserve my objections, but I will not pursue it further until I have a chance to see how everything works in actuality. Sounds fair?

Absolutely. And if I turn out to have been wrong, feel free to say "I told you so."

I think I'd go so far as to say that our aim should be that the skill-based system should replicate V costs and expertise in the first instance. Variants can then use whichever system they like.

Yes, of course. Though I doubt that Vanilla's spell costs and failure rates will match a skill system, at least not one with a simple cost algorithm. We should at least be able to get close though.

[Magnate]

Yes, of course. Though I doubt that Vanilla's spell costs and failure rates will match a skill system, at least not one with a simple cost algorithm. We should at least be able to get close though.

My guess is that where the skill-based algorithm doesn't match the V costs/fail rates, it will be those cases which are already known as anomalous - like the duplicate CMW etc. IOW I don't think we have to match V exactly, warts 'n' all.

[AnonymousHero]

EDIT: http://pastebin.com/VJihZXQG fixed 4-5*-5 and similar expressions

Everything not-completely-crazy should work now

Let's try some more crazy things (I'd suggest adding all the current examples to a little nose test suite -- have I suggested this before? :

+5 => ?
*3 => ?
--7=> ?
sin(10) + abs(5) => ?
sin => ?
sin(sin(sin(sin(3)))) => ?

(Some of these are obviously malformed.)

I still think it's a mistake to conflate evaluation and parsing, but as long as you add a relatively complete test suite (did I mention that already?) any implementation complexity issues or mistakes should be fixable at a later time. Seriously, it's completely trivial to add a few tests.

[Patashu]

+5 => ?
*3 => ?
--7=> ?
sin(10) + abs(5) => ?
sin => ?
sin(sin(sin(sin(3)))) => ?

>>> calculatethis("+5")
5.0
>>> calculatethis("*3")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in calculatethis
ValueError: could not convert string to float: *3
>>> calculatethis("--7")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in calculatethis
ValueError: could not convert string to float: --7
>>> calculatethis("sin(10)+abs(5)")
4.45597888911
>>> calculatethis("sin")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in calculatethis
ValueError: could not convert string to float: sin
>>> calculatethis("sin(sin(sin(sin(3))))")
0.139730046912

The only surprising one is +5, I guess python's string to float parser allows starting with a +

I've never written a test suite in python, will look into it

[Derakon]

I've never written a test suite in python, will look into it

AnonymousHero contributed some nose tests to Pyrel, which you can use as inspiration.

[mtadd]

Tests

AnonymousHero contributed some nose tests to Pyrel, which you can use as inspiration.

Python 2.7 has a quite adequate test suite module builtin. Check out the unittest module. I do recommend installing nosetest and using it as a test runner. For instance, install nosetest and run 'nosetest -v' from the pyrel directory will recursively search for test cases (those derived from uniitest.testcase and nose's testcase base class) and run them. Additionally, it integrates with coverage and can generate nice HTML reports of test case code coverage.

[AnonymousHero]

Python 2.7 has a quite adequate test suite module builtin. Check out the unittest module.

I'm familiar with it and "nose" is fully compatible with test cases based on it.

However, "unittest" is not quite good enough IMO. There are a few things which nose does much better (or which cannot be done with the bog standard "unittest" module):

a) It supports dynamically generated test cases. This is hugely useful because it lets you trivially generate real test cases rather than having to loop *inside* tests. Why is that important, you ask? For one, it lets you use tables for test cases where you still get real test feedback (failed, skipped, passed) for each individual test case. This is something which I specifically wanted for the grammar testing module because I wanted to highlight the presence of skipped test cases. Writing a full test case class (or method) definition for each skipped case would be silly when a simple list of ("input", "expected output") pairs will do.

b) It's a significantly larger amount of typing in the common case because of the "class" definition requirement. Nose requires no such thing -- it'll just find all functions named test_* and run those. (You can still use TestCase-derived classes if you really want all that extra typing.)

c) It requires use of its own special (verbosely named) "assert*" methods instead of just using the standard "assert".

d) More...?

Basically, "unittest" is very JUnit-esque and un-pythonic.

[Patashu]

Ok, I just noticed my code is -really- slow, which is no good for what it's meant to do. I won't post it again until it's faster.

[Mikko Lehtinen]

How are item rarities handled in Pyrel? (The question was inspired by the recent talk about the awesomeness of the Oangband item rarity system.)

[Magnate]

How are item rarities handled in Pyrel? (The question was inspired by the recent talk about the awesomeness of the Oangband item rarity system.)

Base objects (i.e. all consumables and all unenchanted wearables) use the same allocation table as in V. So when generating an item at any given depth, you create an allocation table containing all items which can possibly be generated at that depth, where each item has a number of entries according to its "commonness" rating. So an item with commonness 10 will have ten times as many entries as an item with commonness 1, and is therefore 10x more likely to be chosen. The commonness of items can vary with depth, so an item might have commonness 10 at depths 1 to 50, and commonness 5 at 51 to 75, and then not appear at all below that.

Ego items use the affix-and-theme system, which is based on the one in v4. Affixes also have the commonness-at-depth feature, and this is the basis for their place in the allocation table of affixes available to enchant any given item, but affixes have a lot more flexibility.

1. You can specify which items they may or may not be applied to (at each depth).

2. You can specify global minima and maxima for affixes of each type and goodness rating (i.e. each ego item shall have no more than one "bad" affix and at least one "arcane" affix, or whatever).

3. You can specify explicit conflicts (so the affix "Rusty" will never be applied to anything which already has the affix "Sharp").

Pyrel uses a system of "loot templates" to set out the parameters governing the generation of any particular item. So the "normal" template allows for the creation of any item, but has only a small chance of wearable items having any affixes (slightly higher at deeper levels). The "good" and "great" templates increase the number and quality of affixes.

But templates can do anything. A "food" template could specify that only mushrooms, rations and potions will be allowed in the allocation table. An "orc lair" template specifies that only certain types of Orcish armour and weapons can be generated, and most will have bad or average affixes.

Templates can override the global minima and maxima and the conflicts, so if you want a comedy "GHB" template to drop a Sharp Rusty Dagger of Silliness, you can make it so.

Most monsters will use the normal (/good/great) template most of the time, but some will use others too (e.g. a spellbook template for casters). You'll be able to put local overrides to a template into a particular monster drop profiles - so an archmage will never drop Magic For Beginners.

Dungeon floor items will also use the same system, and will also be able to override template values - so an orc lair at dl99 will have significantly better quality items than the standard orc lair template, while sticking to the same types ... it will presumably have tougher orcs too ...

Sorry, got carried away there. The basic answer to your question is that rarity in pyrel is relative - it depends what else is in the allocation table - i.e. what else can be generated at this depth. We'll use stats to tune the overall appearance of things.

Note that this probably won't apply to artifacts. Pyrel doesn't have artifacts yet, but they'll probably use the v4 system, where the check for artifact is done *before* the base item is chosen, rather than the V system where it's done afterwards. This makes tuning artifact drops much much easier. (Artifacts will still use an allocation table for rarities relative to each other, but they'll have absolute rarity relative to non-artifacts.)

[Mikko Lehtinen]

Thanks, that was very informative!

Sounds like a very flexible system. I'm already imagining implementing Mist's weapon racks, closets and bookshelves with templates...

BTW, it would be nice to collect exactly that kind of explanations for a layman in a wiki. It would be immensely helpful for someone who wants to help with Pyrel or base a variant on it. Or perhaps include explanations in the beginning of text files in lib/edit?

The commonness of items can vary with depth, so an item might have commonness 10 at depths 1 to 50, and commonness 5 at 51 to 75, and then not appear at all below that.

In the Oangband system (also used in Halls of Mist, Sangband, FAangband), you can allocate up to four dungeon level/commonness pairs for base items. For example, rarity 10 for dungeon level 5 and rarity 50 for dungeon level 30. The system will smoothly increase the commonness as the player descends from 5 to 30. Can the Pyrel system do this?

[Derakon]

Thanks, that was very informative!

Sounds like a very flexible system. I'm already imagining implementing Mist's weapon racks, closets and bookshelves with templates...

BTW, it would be nice to collect exactly that kind of explanations for a layman in a wiki. It would be immensely helpful for someone who wants to help with Pyrel or base a variant on it. Or perhaps include explanations in the beginning of text files in lib/edit?

Seconded that this should be on the wiki as a high-level overview of item generation. The current wiki is also good but gets into lower-level details pretty quickly. 1000'-views are also useful.

Unfortunately JSON doesn't allow for in-file explanations since they aren't valid JSON formatting. We're working on documentation for the data files, which will initially live on the wiki and eventually live in the data directory as well (once it stops changing so much).

In the Oangband system (also used in Halls of Mist, Sangband, FAangband), you can allocate up to four dungeon level/commonness pairs for base items. For example, rarity 10 for dungeon level 5 and rarity 50 for dungeon level 30. The system will smoothly increase the commonness as the player descends from 5 to 30. Can the Pyrel system do this?

Pyrel can have any number of allocation rules. They can even overlap; the game will choose the most generous rule. I suppose we could make the "commonness" field be optionally a proc that calculates a value based on depth, if you really wanted to fine-tune things, but that hasn't yet been implemented.

[Mikko Lehtinen]

I suppose we could make the "commonness" field be optionally a proc that calculates a value based on depth, if you really wanted to fine-tune things, but that hasn't yet been implemented.

It's not necessarily just for fine-tuning rarities. I've crafted a very simple but efficient rarity system for a future version of Mist. I'm going to have only six different rarity lines for melee weapons, and perhaps for devices too:

A:0/55:12/30:24/0
A:0/40:24/10:36/10:60/0
A:0/5:12/40:60/0
A:0/0:12/5:24/40:60/10
A:12/0:36/40:60/20
A:36/0:60/70

As long as there are as many melee weapons of each rarity, melee weapons as a group will always be just as common, but your chances of finding better weapons increases slightly every time you take the stairs down.