Various graphs from stats

[Derakon]

Magnate gave me access to the Angband stats SQL database, so I've started doing some basic queries and generating graphs.

First off, the script I'm using to generate these plots is here. Written in Python, dependencies are matplotlib, wx, possibly numpy and/or scipy behind the scenes. Basically ridiculously heavyweight, but that's what I've been using lately and it's easier than writing my own plotting engine.

All of these plots are based on massive numbers of trips through the dungeon, hence why the absolute numbers are so high. I wouldn't pay any attention to absolute numbers; relative differences across levels, and percentages, are more useful.

Potions of Healing:


Stat potions (except Charisma):


Mushrooms of Vigor as a percentage of consumables and wearables found:


Amount of gold found:


Artifacts found:


Let me know if there's something in specific that you'd like to see. As far as I can tell, there isn't a table that gives counts on all items found, so I can't do e.g. Vigor as a percentage of all items, unfortunately. I've also already accidentally nuked Magnate's server once, so I'm trying to avoid complex queries.

[CJNyfalt]

Let me know if there's something in specific that you'd like to see.

Phial distribution by dlvl.

[Derakon]

Phial distribution by dlvl.

That's not really all that helpful because, as I understand it, each level is generated as if no artifacts have been created yet. Thus, there's no "Okay, on this character I found the Phial at dlvl 20, so it's not eligible to be generated any more."

Or at least, that's my understanding, but if that's accurate, why does the Phial count tail off as you get deeper into the dungeon? Magnate?

[d_m]

That's not really all that helpful because, as I understand it, each level is generated as if no artifacts have been created yet. Thus, there's no "Okay, on this character I found the Phial at dlvl 20, so it's not eligible to be generated any more."

I think that was an early design, but at this point I think it handles things correctly (so that the Phial can only be generated once per 100 level run).

[Magnate]

I think that was an early design, but at this point I think it handles things correctly (so that the Phial can only be generated once per 100 level run).

That's correct - artifacts are re-set for each "run" of 100 levels, so the Phial count tails off at higher levels because it's almost always been found before then.

@Derakon: fantastic work - thank you. It's nice to see how smooth lots of the curves are. Even artifacts aren't too ragged (the gradient just needs to come down at bit).

[jens]

Just to see if I understand the graphs correctly: At dlvl 7, 2000 Phials were found over the course of passing that level 100 000 times? So you could go on and say that if you clear dlvl 7 you have a 2% chance to find it :-)

[Magnate]

Just to see if I understand the graphs correctly: At dlvl 7, 2000 Phials were found over the course of passing that level 100 000 times? So you could go on and say that if you clear dlvl 7 you have a 2% chance to find it :-)

Presumably slightly higher, since you might find it a level or three earlier.

The area under the graph should add up to 100k Phials, if it's found every run.

[Derakon]

Either the Phial isn't found every run, or these stats were based on fewer than 100k runs, since I got this graph from having each datapoint be (total Phials found thus far / 100000) * 100:

EDIT: per the below, replaced graph with one scaled to 60k runs.

[Magnate]

Either the Phial isn't found every run, or these stats were based on fewer than 100k runs, since I got this graph from having each datapoint be (total Phials found thus far / 100000) * 100

Ok, that's clearly 60k runs. Due to memory leaks in the stats code (which we may now have fixed), we couldn't do more than 69k runs before it crashed out, and it writes out the data every 10k. I have two other databases of 20k runs each, but as Derakon found out, my server's CPU fan is on the blink (and it's 30c here today) so it overheats if you try to do clever things like add everything together. But I'm confident that the artifact generation is working - that looks a lot like 60k Phials.

[myshkin]

Either the Phial isn't found every run, or these stats were based on fewer than 100k runs, since I got this graph from having each datapoint be (total Phials found thus far / 100000) * 100:

Please note that the metadata table has some useful information, including the number of runs. I haven't looked, but I am guessing that magnate's DB has 60k runs, and that would be consistent with your graph and a 100% drop rate. (We are running into memory problems right around 69k runs, and the code checkpoints every 10k runs by default.)

[Derakon]

OK, I've updated the graph to suit. Thanks for the correction.

[jens]

There's a set of data I'd like to have. The total value of objects, and the total value of monster power, per each dlvl. I.e. I want the data that is used to base the level feelings on.

I'd like either a set of say, 3 values indicating the mean, the lowest 10%, and the highest 10%. Or just the mean with stddev. For both objects and monsters, for each dlvl.

[Magnate]

There's a set of data I'd like to have. The total value of objects, and the total value of monster power, per each dlvl. I.e. I want the data that is used to base the level feelings on.

I'd like either a set of say, 3 values indicating the mean, the lowest 10%, and the highest 10%. Or just the mean with stddev. For both objects and monsters, for each dlvl.

The stats module doesn't currently collect that. It already has a memory footprint of about 1.5GB, and totals of power/value hit the sparse array problem in a big way. If anyone can code up a way to collect this data with minimal memory overhead, I'd happily include it in the stats module.

[jens]

The stats module doesn't currently collect that. It already has a memory footprint of about 1.5GB, and totals of power/value hit the sparse array problem in a big way. If anyone can code up a way to collect this data with minimal memory overhead, I'd happily include it in the stats module.

Aha, I thought you said you used that data to tweak the level feelings? I just felt they might need some more tweaking, so I wanted to check the source...

[Magnate]

Aha, I thought you said you used that data to tweak the level feelings? I just felt they might need some more tweaking, so I wanted to check the source...

The stats module records the level feelings, because they're not a sparse array. So it will show you the distribution of level feelings for all your runs, across the 9 object and 10 monster feelings. But it doesn't record the mon_power or obj_power for each level, or the totals.