can we look at the spell fail percentages

[Nick]

At this point, I have no reason to doubt that the RNG is producing sufficiently random numbers. If someone supplies me with more than anecdotal evidence, I'm happy to look into it in more depth. Specifically, that evidence would have to be something like

• Sufficient data - say 40-50 attempted casts, although 100 or more would be better - with a record of how many attempts it took to cast the spell successfully, and
• Analysis of the results, preferably with a chi-squared test of the observed distribution vs the expected (which should probably be a Poisson distribution).

I intend to point back to this post every time this subject comes up

[Pete Mack]

It's worth noting noting that sequential casts aren't actually sequential RNG rolls, because there are other things going on like noise and random monster generation, as well as cast vs. move choices for LOS monsters. It's exceedingly unlikely that a series of failures is in any way correlated with some periodic misbehavior of the RNG. To be convincing, any analysis has to include an autocorrelation as well, using (say) mod 100 arithmetic to simulate spell-casting.

[PowerWyrm]

What would happen if instead of using the RNG directly for rolling d100 you had it backed up with an array of numbers between 1 and 100, using the RNG to pick up one of these numbers and removing it from the array until the array is empty? You would guarantee that a x% fail spell actually fails x% of the time, but of course you would bias the RNG by giving a straight linear distribution of values. Could be done for spells only if players are actually caring about spell fail rates.

[PowerWyrm]

It's worth noting noting that sequential casts aren't actually sequential RNG rolls, because there are other things going on like noise and random monster generation, as well as cast vs. move choices for LOS monsters. It's exceedingly unlikely that a series of failures is in any way correlated with some periodic misbehavior of the RNG. To be convincing, any analysis has to include an autocorrelation as well, using (say) mod 100 arithmetic to simulate spell-casting.

If you want to see an actual bias in the RNG, just use the Stair Creation spell. I can guarantee you, that deep in the dungeon, the spell will produce an incredible amount of up staircases before you can get a down staircase. And that's supposed to be a coin flip. To counter that, I had to actually code something in PWMAngband that counts the number of up staircases in a row generated by the spell and force placing a down staircase instead after 3-5 unsuccessful tries.

[Pete Mack]

Powerwyrm: if you are always looking for down staircases, and always pick the first one, the expectation is 2x more up than down observed. That said, it's quite possible that there's a bug in the low order bit. But I would not count on it.

[kandrc]

To be fair, a complaint about, specifically, spell fail rates is not necessarily a complaint about the RNG. Even if you're using a state-of-the-art, cryptographically-secure pseudo random number generator, it doesn't mean beans if you're using it incorrectly.

I'm not claiming to believe that there is a problem, just pointing out something that these discussions--which always center on the quality of the PRNG--never seem to address. If there were a bug, usage would be a far more likely place to find it. In a video game, frankly, rand() is more than good enough and discussions about the quality of the PRNG that is actually used are little more than mental masturbation.

[Pete Mack]

Rand() as originally written was explicitly NOT good enough--repeated permutations in the low order bits caused highly non-random behavior. That's why angband has a replacement PRNG in the first place!

[Grotug]

I don't understand any of the technical discussion here, but I would just like to mention that my rods of TO have been succeeding what feels like about 95% of the time over the last two games, even though their fail rate has been around 15%.

I am always surprised at how uneven RNG is. Last game I kept getting !STR, !INT, !DEX, !CON more or less evenly, but I wasn't finding any !WIS. It was odd. Then all of a sudden after all my other stats were maxed I found 5 !WIS on two levels. Still behind the other stats, but it caught up a good deal. My conclusion is that RNG is just really weird in the short term, but does eventually even out to where it's "supposed" to be in the long term.

In Hearthstone people are always complaining the game is rigged. It's funny, because who is the game rigged for? I guess the idea is that people who pay money on cards get better luck drafting an Arena deck or topdecking what they need. As someone who has paid for decks in the game, my luck doesn't seem to be any different than when I was free to play. I tend to have a string of stupidly good luck and a string of stupidly bad luck in Hearthstone as much as I do in Angband. Of course I don't notice or mention the string of normal luck inbetween. :P

[mrfy]

And I note that the last two games the RNG has been kind to me, by generating potions of experience at low levels. In the current game, I discovered one at 650', and promptly went from level 18 to 31.

[AnonymousHero]

I don't understand any of the technical discussion here, but I would just like to mention that my rods of TO have been succeeding what feels like about 95% of the time over the last two games, even though their fail rate has been around 15%.

That's exactly the confirmation/survivorship bias that everbody's talking about. Technical detail aside, all you need to know is that people tend to just remember the outliers, i.e. unusual events, i.e. streaks of either losses or wins. Try to remember all of those times that you didn't get 5 TO successes in a row... Btw, there are a lot of ways to fail 5 TOs in a row (1-S, 1-F, 3-S, 2-S, 1-F, 2-S, etc.). On the other hand there's only one way to have 5 TO attempts succeed. So it's pretty out-of-the-ordinary and thus you remember it).

An example from a BG2 no-reload challenge that I failed: Though it wasn't a critical factor in me failing, I had 4 critical misses in a row. That's something like 1-in-160000 (but see Pete Mack's comment about not counting the first occurrence).

(There are games, especially real-world games like slot machines or similar where there is an actual programmed/built-in win rate that does influence success rates unfairly. That's a whole other discussion, however. Angband really doesn't cheat.)

@Nick (if you happen to read this): I think we agree completely. My reasoning was basically that the original RNG in T2 was something "someone" cooked up in their basement[1] so who knows what it does?!? It's not so much that I thought they might be evil or bad implementers, but I could see no evidence of statistical tests being applied, etc.). So in T2-ah I replaced the RNG with a different RNG that was already pretty highly regarded (PCG). Is Vanilla using a Mersenne Twister these days?

[Pete Mack]

It's also worth noting that there's a slight bias towards shorter failing streaks with TO in particular, because enough failures in a row and you have likely either made a guaranteed escape, or died.

[AnonymousHero]

It's also worth noting that there's a slight bias towards shorter failing streaks with TO in particular, because enough failures in a row and you have likely either made a guaranteed escape, or died.

An excellent point, well made.

I guess I could theoretically claim that I already thought of it under "survivorship bias", but I didn't .

[Grotug]

My rods of TO eventually got down to 14% fail. I never had the dreaded series of fails. In fact, I only failed about 4 times the entire game. I had lots of nice long TO streaks, probably 12 or 15 or so. And I didn't once have two fails in a row. The current game I just finished and my previous game (both lasted to the end) had TO failure rates that seemed to be a fair bit better than 1 out of 7.