Pyrel dev log, part 4

[Magnate]

How are item rarities handled in Pyrel? (The question was inspired by the recent talk about the awesomeness of the Oangband item rarity system.)

Base objects (i.e. all consumables and all unenchanted wearables) use the same allocation table as in V. So when generating an item at any given depth, you create an allocation table containing all items which can possibly be generated at that depth, where each item has a number of entries according to its "commonness" rating. So an item with commonness 10 will have ten times as many entries as an item with commonness 1, and is therefore 10x more likely to be chosen. The commonness of items can vary with depth, so an item might have commonness 10 at depths 1 to 50, and commonness 5 at 51 to 75, and then not appear at all below that.

Ego items use the affix-and-theme system, which is based on the one in v4. Affixes also have the commonness-at-depth feature, and this is the basis for their place in the allocation table of affixes available to enchant any given item, but affixes have a lot more flexibility.

1. You can specify which items they may or may not be applied to (at each depth).

2. You can specify global minima and maxima for affixes of each type and goodness rating (i.e. each ego item shall have no more than one "bad" affix and at least one "arcane" affix, or whatever).

3. You can specify explicit conflicts (so the affix "Rusty" will never be applied to anything which already has the affix "Sharp").

Pyrel uses a system of "loot templates" to set out the parameters governing the generation of any particular item. So the "normal" template allows for the creation of any item, but has only a small chance of wearable items having any affixes (slightly higher at deeper levels). The "good" and "great" templates increase the number and quality of affixes.

But templates can do anything. A "food" template could specify that only mushrooms, rations and potions will be allowed in the allocation table. An "orc lair" template specifies that only certain types of Orcish armour and weapons can be generated, and most will have bad or average affixes.

Templates can override the global minima and maxima and the conflicts, so if you want a comedy "GHB" template to drop a Sharp Rusty Dagger of Silliness, you can make it so.

Most monsters will use the normal (/good/great) template most of the time, but some will use others too (e.g. a spellbook template for casters). You'll be able to put local overrides to a template into a particular monster drop profiles - so an archmage will never drop Magic For Beginners.

Dungeon floor items will also use the same system, and will also be able to override template values - so an orc lair at dl99 will have significantly better quality items than the standard orc lair template, while sticking to the same types ... it will presumably have tougher orcs too ...

Sorry, got carried away there. The basic answer to your question is that rarity in pyrel is relative - it depends what else is in the allocation table - i.e. what else can be generated at this depth. We'll use stats to tune the overall appearance of things.

Note that this probably won't apply to artifacts. Pyrel doesn't have artifacts yet, but they'll probably use the v4 system, where the check for artifact is done *before* the base item is chosen, rather than the V system where it's done afterwards. This makes tuning artifact drops much much easier. (Artifacts will still use an allocation table for rarities relative to each other, but they'll have absolute rarity relative to non-artifacts.)

[Mikko Lehtinen]

How are item rarities handled in Pyrel? (The question was inspired by the recent talk about the awesomeness of the Oangband item rarity system.)

[Patashu]

Ok, I just noticed my code is -really- slow, which is no good for what it's meant to do. I won't post it again until it's faster.

[AnonymousHero]

Python 2.7 has a quite adequate test suite module builtin. Check out the unittest module.

I'm familiar with it and "nose" is fully compatible with test cases based on it.

However, "unittest" is not quite good enough IMO. There are a few things which nose does much better (or which cannot be done with the bog standard "unittest" module):

a) It supports dynamically generated test cases. This is hugely useful because it lets you trivially generate real test cases rather than having to loop *inside* tests. Why is that important, you ask? For one, it lets you use tables for test cases where you still get real test feedback (failed, skipped, passed) for each individual test case. This is something which I specifically wanted for the grammar testing module because I wanted to highlight the presence of skipped test cases. Writing a full test case class (or method) definition for each skipped case would be silly when a simple list of ("input", "expected output") pairs will do.

b) It's a significantly larger amount of typing in the common case because of the "class" definition requirement. Nose requires no such thing -- it'll just find all functions named test_* and run those. (You can still use TestCase-derived classes if you really want all that extra typing.)

c) It requires use of its own special (verbosely named) "assert*" methods instead of just using the standard "assert".

d) More...?

Basically, "unittest" is very JUnit-esque and un-pythonic.

[mtadd]

Tests

AnonymousHero contributed some nose tests to Pyrel, which you can use as inspiration.

Python 2.7 has a quite adequate test suite module builtin. Check out the unittest module. I do recommend installing nosetest and using it as a test runner. For instance, install nosetest and run 'nosetest -v' from the pyrel directory will recursively search for test cases (those derived from uniitest.testcase and nose's testcase base class) and run them. Additionally, it integrates with coverage and can generate nice HTML reports of test case code coverage.

[Derakon]

I've never written a test suite in python, will look into it

AnonymousHero contributed some nose tests to Pyrel, which you can use as inspiration.

[Patashu]

+5 => ?
*3 => ?
--7=> ?
sin(10) + abs(5) => ?
sin => ?
sin(sin(sin(sin(3)))) => ?

>>> calculatethis("+5")
5.0
>>> calculatethis("*3")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in calculatethis
ValueError: could not convert string to float: *3
>>> calculatethis("--7")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in calculatethis
ValueError: could not convert string to float: --7
>>> calculatethis("sin(10)+abs(5)")
4.45597888911
>>> calculatethis("sin")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in calculatethis
ValueError: could not convert string to float: sin
>>> calculatethis("sin(sin(sin(sin(3))))")
0.139730046912

The only surprising one is +5, I guess python's string to float parser allows starting with a +

I've never written a test suite in python, will look into it

[AnonymousHero]

EDIT: http://pastebin.com/VJihZXQG fixed 4-5*-5 and similar expressions

Everything not-completely-crazy should work now

Let's try some more crazy things (I'd suggest adding all the current examples to a little nose test suite -- have I suggested this before? :

+5 => ?
*3 => ?
--7=> ?
sin(10) + abs(5) => ?
sin => ?
sin(sin(sin(sin(3)))) => ?

(Some of these are obviously malformed.)

I still think it's a mistake to conflate evaluation and parsing, but as long as you add a relatively complete test suite (did I mention that already?) any implementation complexity issues or mistakes should be fixable at a later time. Seriously, it's completely trivial to add a few tests.

[Magnate]

Yes, of course. Though I doubt that Vanilla's spell costs and failure rates will match a skill system, at least not one with a simple cost algorithm. We should at least be able to get close though.

My guess is that where the skill-based algorithm doesn't match the V costs/fail rates, it will be those cases which are already known as anomalous - like the duplicate CMW etc. IOW I don't think we have to match V exactly, warts 'n' all.

[Derakon]

I will reserve my objections, but I will not pursue it further until I have a chance to see how everything works in actuality. Sounds fair?

Absolutely. And if I turn out to have been wrong, feel free to say "I told you so."

I think I'd go so far as to say that our aim should be that the skill-based system should replicate V costs and expertise in the first instance. Variants can then use whichever system they like.

Yes, of course. Though I doubt that Vanilla's spell costs and failure rates will match a skill system, at least not one with a simple cost algorithm. We should at least be able to get close though.

[fizzix]

I will reserve my objections, but I will not pursue it further until I have a chance to see how everything works in actuality. Sounds fair?

[Mikko Lehtinen]

Skill-based magic sounds really cool.

In Halls of Mist I'm going to have about the simplest possible skill-based spellcasting system. Learn spells by reading books. True spellcasting classes gain random bonus spells (from either Arcane or Divine list) on certain character levels. To cast any spell you have learned, roll 1d100 <= Spellcasting skill. All spells have a "mastery" level. When you reach that character level, you get to reroll a failed check once.

The exact same system is used for Magic Devices.

[Magnate]

I don't personally think this statement can be supported, but frankly it doesn't matter because we aren't locked into a single system. If you want to have spells that use hardcoded variations depending on the player's class, then you can do that. Just make a different set of procs for determining failure rate / spell cost, and, yes, include the class-specific values in the procs' definitions.

This is just brilliant.

I daresay most spells can simply use the skill-based system instead and only a few exceptions, if any, will need the more specific approach.

I think I'd go so far as to say that our aim should be that the skill-based system should replicate V costs and expertise in the first instance. Variants can then use whichever system they like.

I just realised how easy it will be to reimplement Sangband in Pyrel ....

[Magnate]

Sit down at home, review Calculator.cs for C#, hammer out bug fixes for 2 hours \o/ (Most mistakes were from not being used to python, a few were forgetting important things like the lookaheads/lookbehinds on unary operators so 2V2 doesn't turn into 21.414etc, and getting order of operations right for operators of the same precedence)

http://pastebin.com/20vCSF7Y

Everything not-completely-crazy should work now


>>> calculatethis("(8+5)*3")
39.0
>>> calculatethis("5+abs(5*abs(-5))")
30.0
>>>

Thanks Patashu - that is just fantastic, and perfect timing for me. After my discussion with Derakon I had moved to the point where I was just about to suggest using a 3rd-party library, but now we can try to avoid that.

[Patashu]

Sit down at home, review Calculator.cs for C#, hammer out bug fixes for 2 hours \o/ (Most mistakes were from not being used to python, a few were forgetting important things like the lookaheads/lookbehinds on unary operators so 2V2 doesn't turn into 21.414etc, and getting order of operations right for operators of the same precedence)

EDIT: http://pastebin.com/H07vMSEf fixed 4-5*-5 and similar expressions

Everything not-completely-crazy should work now

Given your evaluator, what is the result of "(8+5)*3"? Or "5+abs(5*abs(-5))"?

>>> calculatethis("(8+5)*3")
39.0
>>> calculatethis("5+abs(5*abs(-5))")
30.0
>>>