Randomness and Analytics of Die Rolls

[Weissrolf]

Both could be interesting, same dice per player and same dice by all players. I don't know if dice are calculated by the player client or by the server when a player rolls?

It could also be useful to see dice rolls (or totals) on a time-graph, so that we get a better picture of the distribution in time.

[damned]

When you say useful what do you mean?
True randomness will plenty of times give you results that dont look random.

[Weissrolf]

Let me ask a question in return: what seed/source is used for FG's random number calculation?

[damned]

https://www.fantasygrounds.com/forums/showthread.php?37630-RNG-Issues-Nothing-but-ones

[Weissrolf]

There is no random number generator. The dice are a physics simulation of a perfectly weighted dice dropped from a fixed height onto a flat surface.
No random generator, so no seed. In Classic the dice move while in the air, in Unity it does not seem so (which looks strange and suggest an awkward dice throwing technique).


The dice is given a random rotation based on the computer RNG every time it is picked up or rolled.
A random generator, what is the seed and how is the RNG (Randon Number Generator) done?


Also, the lateral direction and power of the die roll is determined by the speed of the mouse prior to the die release.
So the seed for lateral direction and power is determined by mouse-movement.


If the speed is too slow, the speed is increased to a minimum. If you double click instead of rolling with the mouse, then a random direction is chosen with a random power based on the computer RNG.
Except when it isn't (we mostly use double-clicking here), then a random generator is used front to back, the seed and implementation of which I wonder about.

[damned]

No computer generated RNG is ever truly truly random. That does not mean that for all practical purposes the numbers are not random.

[Weissrolf]

Yes, thanks, I knew that, it depends on the algorithm and seed being used. Do you happen to know what seed is used? I assume a pseudo-random seed (unless something like random.org is used), but the question still stands.

[damned]

Yes, thanks, I knew that, it depends on the algorithm and seed being used. Do you happen to know what seed is used? I assume a pseudo-random seed (unless something like random.org is used), but the question still stands.

Ive directed you to the statement made by the devs.

[Weissrolf]

Yes, thank you. I quoted said statement ("There is no RNG, but there is a RNG") and it does not tell us anything about the seed.

So we are back at were we started, the question unfortunately was not answered, not even with a "we don't know and the devs won't tell". Maybe the answer is hidden somewhere in this forum, but I was not able to find it yet. This means we can only look at the RNG outcome, which this thread is about. Would have been nice to have more information on the process available, though.

[damned]

I think that the statement made by the devs is as much info as they are prepared to release at this time. They have made the same statement at least a half dozen times.

[Weissrolf]

Then back to your question:


When you say useful what do you mean?
Useful in understanding/analyzing the result of FG's random number generator while the devs are not willing to share the underlying process (such as which RNG seed is being used).

[seycyrus]

Then back to your question:


Useful in understanding/analyzing the result of FG's random number generator while the devs are not willing to share the underlying process (such as which RNG seed is being used).

Why don't you just do a statistical analysis on a large number of rolls and then we can discuss your results to your hearts content.

[Weissrolf]

What are your thoughts on having a semi-automated way to build and display game statistics for each of your campaigns?
Yes, let's do that.

[seycyrus]

Yes, let's do that.

What are your answers to the other questions in that post?

[Weissrolf]

1. Yes.
2. No. Maybe half as much.

[damned]

I am in the process of (trying to) coding a MoreCore roll so people who like numbers can run some tests.
Its not working properly yet but i had it roll a d6 600x consecutively and this is what I got.

Plenty of 3 in a row - as best as I can tell there was a string of four 4s and nothing longer.
lowest number rolled was #3 with 86
highest number rolled was #2 with 108
6s outrolled 1s by the merest amount

https://fantasygrounds.com/forums/attachment.php?attachmentid=40725

40725

[Weissrolf]

There is a row of 5 times rolled 4 in there. ;)

Are you sure that these are 600 rolls and not 601? I just typed them all into Excel for calculations, but got an odd number. So before starting all over again, maybe your extension counts from 0 to 600 instead of 1 to 600? If not then I have to check all numbers again.

[Weissrolf]

I found the wrongly entered row and I am now in the process of manually typing the correct numbers back in (or rather shifting everything by one place).

[Weissrolf]

The data-size of 600 rolls is still too small, but here are the calculations I am more interested in than just averages.

Looking at two consecutive rolls as 2D6 we count 300 rolls with an expected and measured average of 7.

There is an average chance of 1/6 (16.67%) to roll a double with 2D6 (2 consecutive rolls), this corresponds to 50 out of 300 rolls.
There is an average chance of 1/36 (2.78%) to roll a specific double (1s, 2s, 3s, 4s, 5s, 6s), this corresponds to 8.3 out of 300 rolls.

Number of doubles rolled: 67

1s: 10
2s: 13
3s: 6
4s: 16
5s: 9
6s: 12

Ignoring the first and last roll shifts the results by 1 step (2nd + 3rd roll = 2d6 instead of 1st + 2nd roll). This creates a total of 298 rolls, the calculated average is 6.98.

Number of doubles rolled: 47

1s: 4
2s: 13
3s: 2
4s: 11
5s: 7
6s: 10

With a larger data-set (more rolls) the number of doubles should average towards 50 and the number of each specific double should trend towards 8.3. So we need more rolls and a way to automatically calculate these statistics, because I am not going to do it manually again, like never ever. :P

Another interesting metric would be the time distribution of numbers to see if we see patterns of ups and downs of averages. No use in knowing that the average over all numbers is 7 when we might get regular (patterns of) "locked" highs and lows. As with the doubles statistics above this is meant to reveal if rolls are affected by their predecessor rolls.

[Weissrolf]

Three anecdotes concerning computer generated random numbers:

- I once tested several tone/noise generators (software) running on the same Windows hardware. Only one Japanese program was able to create brown/pink/white noise being properly distributed over all frequency bands. One could feed dice results into a tone generator and easily measure the distribution (maybe even hear it when it's not as much noise as it should be).

- On the Commodore Amiga one could use the current position of the displays cathode ray as seed for RNG operations.

- The game "Little Computer People" allowed to enter a seed number that affected how the life of the little guy and his dog would unfold. Some Solitaire card games also allow to enter a seed number manually.

https://www.retrogamer.net/wp-content/uploads/2013/10/lcp.png

[damned]

Some people like numbers waaaay too much!
I am having some issues finishing the coding on this roll.
Once that works you can roll bigger data sets.

It will also display a simple % measure of each number rolled but I have no idea if I can calculate those other stats you have done.

[Weissrolf]

Doubles can easily be calculated, just subtract one roll from the other (=0). ;)

[damned]

I set it up to roll 10,000 d6 and this is the result

#1 | #1685 | s'16%'
#2 | #1721 | s'17%'
#3 | #1651 | s'16%'
#4 | #1818 | s'18%'
#5 | #1557 | s'15%'
#6 | #1568 | s'15%'
#7 | #10000 | s'100%'



s'sResults' | s'6, 1, 5, 6, 1, 5, 6, 1, 2, 2, 2, 3, 2, 1, 4, 1, 4, 4, 1, 4, 5, 1, 2, 2, 1, 1, 3, 6, 3, 6, 6, 5, 2, 3, 3, 2, 4, 5, 2, 4, 5, 6, 5, 4, 6, 5, 6, 4, 2, 4, 4, 6, 2, 1, 6, 5, 4, 5, 4, 4, 4, 4, 3, 2, 6, 4, 6, 4, 6, 6, 2, 1, 3, 5, 4, 5, 5, 3, 1, 2, 3, 2, 1, 3, 4, 6, 2, 6, 2, 5, 1, 6, 4, 5, 4, 5, 2, 3, 1, 4, 6, 2, 6, 3, 6, 6, 1, 2, 2, 3, 6, 1, 4, 1, 2, 5, 5, 4, 6, 5, 5, 4, 2, 6, 2, 4, 3, 2, 2, 6, 3, 5, 2, 5, 4, 4, 5, 3, 1, 1, 6, 4, 2, 4, 4, 3, 1, 1, 3, 5, 5, 2, 1, 5, 1, 1, 6, 1, 2, 1, 6, 3, 1, 4, 5, 3, 6, 3, 2, 6, 2, 6, 4, 3, 1, 1, 1, 4, 6, 4, 4, 4, 2, 2, 1, 1, 1, 6, 6, 5, 3, 4, 4, 2, 3, 3, 1, 3, 5, 4, 4, 4, 1, 4, 1, 5, 6, 3, 6, 1, 2, 3, 1, 1, 2, 2, 2, 6, 4, 1, 5, 3, 3, 1, 4, 2, 6, 6, 6, 4, 2, 5, 3, 4, 3, 5, 3, 1, 4, 5, 1, 4, 1, 1, 6, 5, 1, 6, 2, 1, 4, 6, 4, 2, 6, 1, 3, 5, 1, 3, 4, 1, 1, 1, 4, 4, 2, 6, 4, 2, 4, 3, 3, 2, 1, 3, 4, 4, 4, 4, 6, 4, 3, 4, 6, 2, 4, 6, 1, 1, 3, 4, 2, 5, 5, 2, 4, 3, 2, 4, 2, 1, 3, 3, 2, 2, 5, 5, 6, 4, 3, 1, 6, 2, 4, 5, 1, 6, 6, 5, 4, 1, 4, 2, 5, 4, 4, 5, 5, 1, 5, 4, 2, 2, 6, 1, 3, 3, 3, 2, 3, 4, 4, 3, 6, 3, 2, 4, 5, 2, 6, 4, 1, 5, 2, 4, 4, 6, 5, 2, 1, 2, 5, 2, 4, 5, 3, 6, 1, 1, 5, 5, 3, 4, 3, 6, 4, 5, 1, 4, 6, 5, 1, 1, 4, 2, 3, 4, 5, 4, 5, 4, 3, 4, 1, 4, 4, 2, 2, 4, 4, 6, 3, 6, 2, 5, 4, 3, 2, 4, 4, 4, 2, 4, 5, 2, 6, 2, 1, 6, 2, 1, 1, 5, 6, 6, 5, 3, 4, 1, 2, 4, 6, 6, 3, 5, 1, 5, 6, 3, 6, 1, 3, 5, 4, 2, 5, 4, 4, 6, 5, 3, 2, 5, 5, 6, 1, 2, 3, 4, 4, 3, 3, 5, 5, 2, 3, 2, 2, 1, 3, 1, 1, 4, 4, 1, 5, 2, 4, 5, 3, 4, 5, 1, 1, 5, 3, 2, 3, 3, 1, 6, 4, 1, 5, 1, 4, 5, 4, 1, 6, 4, 2, 4, 5, 3, 5, 1, 6, 2, 3, 1, 5, 6, 4, 5, 6, 5, 2, 3, 6, 1, 3, 4, 5, 3, 6, 2, 4, 1, 1, 6, 1, 2, 5, 5, 6, 3, 4, 3, 4, 4, 3, 2, 4, 4, 4, 2, 1, 5, 4, 1, 4, 4, 4, 6, 3, 3, 3, 4, 2, 6, 3, 2, 4, 1, 6, 6, 4, 5, 6, 2, 2, 5, 5, 3, 4, 6, 4, 1, 4, 2, 5, 2, 3, 2, 4, 4, 6, 3, 5, 6, 2, 1, 6, 4, 6, 5, 3, 5, 6, 4, 3, 1, 1, 1, 5, 5, 3, 6, 1, 3, 4, 2, 5, 1, 3, 1, 4, 2, 3, 1, 6, 4, 4, 2, 6, 1, 4, 2, 2, 6, 2, 5, 1, 1, 2, 4, 4, 2, 3, 1, 3, 6, 2, 1, 2, 2, 6, 3, 5, 4, 2, 6, 3, 1, 6, 6, 2, 6, 5, 2, 2, 5, 5, 4, 5, 3, 6, 2, 2, 4, 1, 5, 2, 6, 5, 3, 4, 6, 2, 4, 2, 3, 4, 3, 5, 4, 2, 2, 4, 1, 2, 2, 6, 3, 1, 2, 2, 5, 1, 6, 1, 4, 5, 6, 1, 4, 2, 3, 2, 3, 2, 4, 4, 4, 3, 5, 6, 1, 3, 2, 6, 3, 6, 3, 3, 2, 2, 1, 2, 5, 4, 6, 4, 4, 1, 3, 5, 2, 4, 1, 3, 4, 2, 3, 5, 4, 6, 6, 1, 6, 5, 3, 1, 6, 4, 6, 6, 5, 4, 6, 5, 5, 2, 3, 5, 6, 2, 2, 3, 6, 4, 1, 4, 3, 4, 6, 3, 6, 2, 5, 4, 6, 4, 6, 1, 1, 1, 1, 5, 6, 2, 2, 2, 2, 4, 5, 1, 5, 3, 1, 5, 6, 5, 1, 5, 6, 1, 1, 6, 5, 6, 5, 5, 3, 6, 1, 4, 1, 3, 5, 5, 1, 5, 4, 4, 1, 4, 4, 4, 5, 3, 4, 4, 5, 6, 5, 5, 2, 1, 6, 6, 6, 5, 6, 4, 1, 5, 5, 5, 1, 6, 5, 5, 4, 6, 6, 1, 5, 2, 4, 3, 3, 6, 3, 4, 5, 1, 5, 5, 2, 5, 2, 5, 4, 4, 3, 2, 5, 5, 1, 4, 6, 1, 3, 3, 3, 5, 5, 4, 1, 6, 1, 3, 3, 2, 1, 5, 4, 4, 2, 2, 6, 4, 5, 6, 2, 1, 3, 3, 6, 3, 6, 2, 1, 1, 2, 5, 2, 3, 4, 4, 5, 4, 4, 1, 4, 4, 2, 6, 3, 4, 4, 4, 6, 3, 5, 6, 4, 6, 5, 3, 3, 2, 2, 1, 3, 1, 5, 4, 4, 6, 1, 5, 4, 5, 4, 4, 5, 1, 6, 3, 4, 5, 1, 5, 2, 3, 4, 5, 4, 3, 5, 4, 1, 1, 1, 5, 2, 4, 6, 5, 5, 5, 2, 3, 2, 5, 6, 1, 2, 3, 5, 4, 3, 1, 5, 6, 4, 3, 4, 3, 1, 4, 3, 5, 2, 2, 1, 1, 3, 3, 2, 2, 6, 4, 4, 5, 3, 5, 5, 1, 6, 1, 4, 6, 2, 6, 6, 1, 1, 3, 2, 1, 6, 1, 3, 3, 2, 3, 4, 6, 1, 5, 3, 4, 4, 3, 1, 3, 6, 1, 1, 6, 1, 2, 6, 6, 4, 4, 2, 4, 5, 1, 5, 5, 1, 1, 1, 3, 6, 6, 1, 4, 2, 2, 1, 3, 4, 1, 5, 2, 4, 4, 4, 2, 2, 1, 6, 4, 5, 5, 5, 4, 6, 1, 6, 1, 4, 2, 4, 6, 2, 6, 4, 2, 1, 4, 3, 5, 3, 1, 3, 5, 4, 4, 4, 5, 3, 4, 6, 3, 1, 4, 5, 2, 6, 6, 3, 2, 3, 5, 3, 4, 1, 1, 1, 2, 1, 3, 4, 4, 3, 1, 5, 3, 6, 4, 4, 2, 6, 4, 4, 3, 5, 1, 5, 3, 6, 3, 1, 3, 6, 6, 6, 1, 5, 4, 1, 4, 1, 1, 2, 3, 4, 4, 3, 6, 5, 6, 6, 2, 6, 2, 3, 1, 1, 5, 6, 5, 2, 1, 6, 4, 2, 2, 6, 2, 2, 2, 3, 2, 5, 5, 6, 5, 5, 4, 6, 2, 1, 2, 2, 2, 4, 1, 2, 3, 2, 5, 6, 1, 5, 3, 5, 3, 3, 4, 4, 2, 2, 2, 3, 4, 6, 2, 1, 3, 5, 5, 2, 6, 5, 5, 2, 6, 6, 4, 1, 1, 3, 2, 3, 2, 1, 1, 2, 4, 3, 4, 4, 3, 6, 6, 2, 5, 6, 3, 4, 5, 6, 6, 3, 1, 3, 6, 1, 1, 6, 4, 3, 5, 3, 4, 4, 4, 1, 4, 1, 6, 2, 3, 5, 2, 4, 3, 5, 6, 4, 4, 5, 6, 6, 3, 6, 2, 4, 5, 2, 2, 3, 4, 4, 6, 4, 2, 4, 3, 2, 1, 2, 5, 2, 6, 5, 1, 2, 1, 1, 1, 1, 4, 3, 5, 3, 3, 1, 5, 1, 1, 2, 1, 4, 3, 6, 6, 4, 5, 5, 3, 2, 2, 4, 5, 6, 2, 6, 1, 1, 6, 2, 1, 6, 4, 1, 5, 6, 3, 2, 5, 3, 4, 4, 6, 5, 6, 5, 4, 1, 3, 5, 5, 2, 4, 4, 5, 3, 6, 1, 1, 6, 1, 2, 5, 5, 1, 2, 4, 1, 6, 2, 2, 2, 1, 3, 4, 1, 6, 3, 6, 2, 4, 6, 5, 3, 1, 2, 1, 6, 4, 4, 4, 6, 6, 4, 5, 6, 4, 3, 4, 2, 2, 2, 3, 2, 6, 6, 3, 4, 6, 4, 2, 4, 6, 1, 3, 5, 2, 6, 6, 5, 4, 3, 1, 5, 2, 5, 3, 3, 1, 6, 5, 1, 1, 4, 5, 1, 6, 2, 1, 4, 6, 5, 3, 1, 5, 2, 4, 6, 3, 4, 1, 5, 2, 5, 5, 3, 2, 3, 5, 1, 2, 5, 3, 1, 6, 5, 1, 4, 3, 5, 5, 1, 3, 5, 4, 3, 4, 6, 4, 6, 5, 3, 6, 4, 4, 6, 2, 6, 5, 5, 4, 4, 1, 6, 6, 4, 5, 5, 2, 4, 1, 1, 5, 3, 6, 5, 4, 1, 1, 1, 5, 4, 2, 3, 4, 2, 1, 1, 5, 1, 1, 2, 2, 3, 3, 4, 4, 3, 4, 4, 1, 2, 5, 3, 5, 2, 6, 4, 1, 1, 1, 4, 3, 6, 4, 5, 6, 6, 4, 1, 4, 6, 4, 6, 3, 6, 3, 5, 6, 3, 5, 2, 3, 5, 6, 3, 4, 2, 6, 1, 1, 3, 2, 4, 1, 5, 5, 1, 4, 6, 1, 4, 6, 3, 6, 3, 6, 4, 4, 1, 5, 2, 4, 5, 5, 2, 3, 4, 2, 1, 4, 6, 4, 3, 3, 3, 2, 2, 1, 2, 2, 5, 6, 3, 6, 1, 1, 2, 5, 6, 4, 5, 6, 1, 5, 6, 4, 5, 2, 4, 1, 6, 4, 5, 4, 2, 5, 4, 6, 6, 3, 5, 4, 3, 5, 5, 2, 5, 1, 6, 3, 1, 3, 1, 1, 1, 4, 2, 1, 2, 1, 1, 6, 6, 4, 6, 3, 3, 2, 6, 5, 4, 2, 4, 6, 2, 5, 4, 3, 5, 6, 2, 4, 1, 6, 1, 3, 2, 6, 5, 5,

[damned]

1, 2, 2, 1, 4, 6, 4, 2, 2, 2, 2, 5, 6, 3, 4, 3, 5, 2, 3, 2, 1, 3, 6, 4, 2, 1, 2, 2, 4, 5, 6, 5, 4, 5, 6, 1, 6, 1, 2, 2, 5, 3, 3, 3, 2, 6, 5, 3, 1, 2, 5, 6, 1, 5, 3, 5, 1, 5, 6, 4, 6, 1, 3, 3, 2, 3, 5, 4, 1, 2, 6, 2, 4, 5, 6, 6, 3, 6, 4, 5, 4, 2, 3, 3, 6, 5, 4, 6, 5, 3, 5, 4, 2, 2, 5, 6, 5, 4, 4, 5, 1, 4, 1, 3, 3, 4, 4, 4, 6, 4, 6, 6, 3, 5, 5, 1, 2, 5, 6, 3, 6, 5, 1, 2, 2, 5, 2, 5, 2, 5, 4, 4, 5, 6, 1, 2, 6, 3, 4, 6, 4, 1, 5, 5, 6, 6, 4, 2, 1, 4, 3, 3, 2, 3, 1, 1, 1, 4, 3, 1, 2, 6, 4, 5, 3, 3, 4, 4, 1, 5, 3, 6, 2, 2, 2, 3, 4, 4, 1, 5, 1, 5, 5, 1, 1, 4, 5, 3, 3, 6, 1, 6, 5, 3, 6, 1, 3, 3, 5, 2, 2, 6, 2, 1, 4, 4, 3, 5, 6, 5, 1, 1, 6, 5, 5, 3, 5, 5, 3, 2, 5, 1, 3, 4, 2, 2, 5, 3, 4, 1, 3, 3, 5, 4, 5, 4, 5, 5, 6, 2, 1, 1, 3, 6, 2, 2, 4, 2, 6, 1, 3, 2, 6, 5, 4, 5, 2, 4, 4, 5, 5, 1, 5, 5, 1, 3, 1, 2, 5, 1, 3, 1, 4, 5, 1, 1, 4, 1, 3, 3, 1, 6, 3, 5, 3, 4, 4, 2, 3, 1, 4, 3, 5, 1, 4, 3, 1, 2, 4, 4, 6, 3, 1, 6, 3, 3, 4, 3, 4, 3, 1, 4, 4, 5, 2, 4, 4, 3, 5, 3, 6, 2, 3, 3, 4, 2, 5, 2, 3, 5, 3, 1, 6, 2, 1, 5, 1, 1, 3, 4, 3, 3, 1, 5, 4, 1, 3, 6, 4, 6, 6, 5, 4, 1, 4, 5, 3, 5, 4, 5, 1, 5, 6, 4, 1, 3, 2, 4, 1, 6, 6, 2, 6, 3, 5, 5, 4, 6, 1, 2, 4, 1, 4, 2, 5, 5, 1, 2, 6, 6, 1, 5, 3, 5, 1, 1, 1, 5, 2, 4, 3, 5, 5, 5, 6, 6, 6, 1, 5, 5, 4, 5, 3, 2, 5, 6, 6, 2, 4, 1, 4, 3, 4, 4, 6, 5, 4, 5, 1, 6, 1, 3, 2, 1, 1, 5, 4, 3, 2, 5, 3, 3, 1, 1, 5, 4, 3, 5, 6, 6, 5, 6, 4, 1, 3, 2, 3, 4, 2, 5, 6, 4, 1, 3, 6, 2, 2, 3, 4, 3, 2, 3, 6, 4, 1, 2, 5, 2, 1, 6, 3, 2, 2, 6, 6, 6, 6, 2, 3, 4, 1, 4, 6, 5, 2, 3, 6, 2, 6, 3, 1, 1, 3, 1, 5, 3, 6, 4, 6, 1, 3, 6, 6, 6, 5, 5, 5, 6, 1, 6, 3, 4, 3, 6, 1, 2, 3, 4, 6, 4, 1, 4, 1, 2, 1, 1, 3, 4, 2, 6, 6, 4, 4, 3, 3, 3, 1, 1, 1, 5, 1, 1, 3, 2, 6, 3, 5, 1, 1, 1, 1, 5, 2, 2, 3, 4, 5, 4, 3, 6, 1, 2, 2, 5, 5, 1, 6, 4, 2, 6, 2, 1, 2, 1, 2, 1, 4, 3, 5, 3, 6, 4, 6, 1, 2, 3, 4, 1, 5, 6, 3, 5, 1, 3, 4, 4, 6, 5, 3, 4, 3, 6, 6, 3, 5, 2, 2, 5, 5, 1, 5, 3, 6, 5, 5, 6, 5, 3, 5, 3, 5, 6, 5, 1, 2, 1, 4, 6, 1, 2, 1, 6, 1, 1, 6, 1, 2, 2, 6, 6, 4, 2, 2, 4, 2, 1, 4, 6, 3, 5, 4, 3, 2, 4, 2, 6, 6, 4, 2, 4, 2, 3, 3, 5, 3, 6, 5, 5, 4, 4, 5, 3, 1, 6, 2, 6, 5, 4, 2, 5, 4, 5, 5, 5, 2, 2, 1, 1, 5, 4, 2, 1, 1, 2, 5, 4, 5, 5, 3, 6, 5, 6, 4, 5, 3, 4, 5, 4, 6, 5, 5, 3, 2, 2, 6, 1, 6, 5, 5, 5, 6, 6, 4, 6, 4, 4, 4, 6, 3, 6, 2, 2, 6, 3, 4, 2, 5, 6, 3, 4, 3, 4, 1, 6, 3, 2, 4, 3, 2, 5, 1, 5, 6, 4, 1, 5, 3, 1, 4, 3, 4, 1, 5, 3, 6, 5, 5, 3, 5, 5, 1, 2, 3, 4, 2, 5, 1, 4, 5, 1, 5, 3, 1, 1, 1, 6, 1, 5, 5, 3, 2, 3, 5, 5, 3, 3, 6, 5, 6, 6, 2, 5, 1, 4, 4, 1, 3, 5, 4, 6, 6, 1, 5, 6, 4, 4, 2, 5, 2, 6, 3, 3, 2, 5, 2, 1, 3, 5, 6, 2, 6, 1, 4, 2, 5, 6, 5, 5, 3, 3, 4, 3, 4, 3, 3, 2, 4, 4, 6, 1, 2, 3, 1, 2, 1, 1, 5, 4, 4, 6, 2, 6, 6, 3, 5, 6, 6, 5, 4, 3, 3, 6, 3, 2, 4, 6, 2, 4, 5, 4, 4, 6, 4, 1, 6, 3, 3, 1, 3, 2, 2, 6, 3, 6, 5, 6, 5, 5, 4, 4, 3, 2, 2, 5, 5, 5, 2, 2, 6, 3, 5, 6, 5, 1, 5, 6, 6, 6, 2, 3, 6, 1, 2, 6, 6, 2, 5, 4, 4, 3, 2, 5, 2, 3, 5, 4, 4, 4, 1, 6, 4, 4, 2, 5, 5, 3, 6, 4, 5, 3, 4, 1, 3, 3, 3, 2, 2, 1, 3, 3, 2, 1, 2, 1, 3, 5, 1, 4, 1, 2, 5, 1, 3, 3, 2, 4, 4, 4, 3, 5, 4, 4, 2, 5, 6, 3, 3, 6, 2, 6, 4, 1, 3, 6, 4, 5, 3, 3, 3, 3, 5, 4, 5, 6, 2, 1, 6, 3, 3, 2, 3, 4, 3, 5, 2, 5, 2, 5, 1, 4, 4, 6, 6, 4, 1, 4, 1, 6, 2, 3, 5, 4, 1, 6, 5, 2, 3, 1, 1, 4, 6, 4, 4, 2, 2, 2, 1, 5, 3, 1, 6, 2, 2, 4, 2, 3, 2, 5, 3, 3, 5, 4, 1, 1, 3, 4, 3, 5, 5, 2, 3, 5, 1, 3, 1, 2, 6, 3, 5, 3, 4, 1, 6, 2, 4, 5, 3, 5, 3, 2, 2, 2, 4, 4, 3, 4, 3, 3, 5, 6, 5, 3, 4, 6, 4, 3, 5, 4, 4, 4, 4, 6, 2, 4, 2, 5, 3, 5, 2, 5, 4, 2, 5, 4, 2, 1, 1, 3, 3, 1, 1, 2, 6, 5, 3, 4, 1, 3, 3, 2, 2, 2, 3, 4, 4, 5, 6, 4, 4, 6, 6, 1, 6, 6, 6, 6, 2, 3, 6, 6, 1, 2, 5, 4, 2, 4, 1, 4, 2, 2, 5, 4, 4, 5, 5, 2, 6, 2, 2, 4, 6, 2, 2, 6, 5, 4, 6, 6, 6, 4, 2, 6, 5, 6, 2, 6, 3, 6, 1, 1, 4, 1, 2, 5, 4, 3, 2, 4, 6, 6, 3, 4, 4, 3, 5, 3, 5, 2, 4, 3, 1, 4, 2, 1, 4, 3, 5, 1, 5, 3, 1, 3, 2, 4, 2, 1, 6, 5, 5, 1, 2, 4, 4, 4, 6, 2, 4, 6, 6, 1, 1, 2, 3, 1, 3, 4, 1, 3, 6, 5, 2, 3, 4, 6, 1, 5, 3, 5, 4, 6, 2, 6, 4, 3, 3, 3, 4, 5, 3, 2, 1, 2, 5, 6, 6, 1, 3, 3, 1, 2, 4, 3, 5, 1, 6, 2, 3, 2, 5, 3, 6, 4, 6, 2, 2, 4, 4, 6, 1, 4, 2, 6, 2, 6, 3, 3, 1, 3, 5, 3, 1, 4, 3, 3, 4, 2, 5, 3, 4, 3, 3, 1, 5, 1, 1, 3, 4, 5, 4, 2, 5, 2, 4, 4, 3, 2, 3, 6, 4, 4, 4, 1, 3, 6, 4, 6, 5, 1, 3, 2, 1, 5, 1, 5, 3, 2, 3, 1, 6, 6, 5, 4, 1, 2, 6, 1, 4, 1, 6, 4, 5, 3, 3, 4, 1, 4, 3, 2, 5, 6, 1, 2, 3, 2, 5, 5, 6, 5, 5, 2, 5, 2, 5, 3, 6, 2, 2, 4, 5, 6, 4, 1, 5, 4, 6, 2, 1, 2, 4, 1, 5, 3, 2, 1, 1, 2, 4, 4, 4, 1, 1, 2, 4, 5, 1, 6, 4, 2, 1, 1, 2, 1, 2, 4, 2, 2, 5, 4, 6, 2, 3, 1, 1, 5, 4, 4, 4, 6, 6, 1, 3, 3, 3, 5, 1, 1, 4, 1, 3, 2, 1, 4, 3, 2, 3, 5, 5, 6, 2, 3, 3, 2, 3, 6, 2, 1, 4, 2, 2, 5, 1, 4, 4, 2, 4, 1, 2, 5, 4, 4, 2, 3, 5, 4, 2, 2, 3, 5, 4, 4, 5, 5, 6, 1, 3, 4, 3, 5, 6, 4, 5, 5, 3, 3, 5, 3, 1, 6, 4, 2, 2, 2, 5, 2, 1, 5, 5, 1, 1, 3, 1, 1, 5, 1, 4, 1, 3, 5, 5, 3, 3, 5, 5, 6, 3, 3, 2, 1, 6, 3, 5, 3, 1, 3, 3, 5, 4, 2, 3, 6, 5, 6, 4, 3, 1, 3, 3, 4, 3, 1, 4, 1, 3, 3, 2, 4, 1, 1, 6, 3, 5, 2, 3, 5, 6, 6, 1, 1, 5, 3, 1, 1, 2, 5, 5, 5, 5, 1, 3, 3, 5, 2, 2, 2, 4, 3, 5, 1, 1, 1, 4, 1, 6, 1, 4, 5, 5, 3, 6, 4, 3, 1, 2, 5, 2, 4, 5, 3, 3, 5, 1, 4, 3, 1, 3, 3, 1, 5, 1, 1, 1, 4, 2, 6, 2, 6, 2, 6, 5, 1, 3, 2, 5, 6, 2, 6, 5, 1, 4, 6, 2, 1, 3, 1, 3, 5, 2, 6, 6, 2, 6, 5, 6, 2, 6, 5, 1, 6, 1, 2, 6, 1, 6, 5, 4, 2, 3, 2, 5, 1, 1, 2, 2, 4, 3, 2, 5, 3, 1, 5, 4, 1, 2, 3, 5, 6, 1, 1, 4, 6, 5, 6, 6, 2, 3, 3, 3, 6, 3, 2, 2, 3, 2, 3, 4, 5, 3, 3, 2, 4, 6, 5, 3, 4, 5, 3, 2, 1, 4, 2, 1, 6, 4, 3, 6, 3, 2, 5, 4, 2, 1, 5, 5, 4, 3, 5, 4, 2, 6, 1, 5, 6, 5, 2, 6, 6, 1, 2, 2, 2, 2, 2, 6, 3, 1, 3, 3, 5, 6, 5, 6, 2, 6, 1, 2, 4, 1, 2, 4, 5, 1, 3, 6, 3, 6, 4, 6, 2, 3, 6, 6, 5, 1, 4, 6, 1, 6, 4, 6, 3, 4, 3, 3, 5, 2, 2, 5, 2, 6, 2, 1, 2, 6, 1, 1, 6, 6, 1, 4, 5, 6, 4, 6, 6, 3, 2, 4, 4, 6, 2, 2, 1, 5, 5, 6, 4, 3, 3, 2, 6, 2, 2, 1, 3, 3, 2, 3, 3, 1, 3, 6, 1, 6, 5, 4, 4, 4, 3, 3, 1, 6, 3, 6, 6, 3, 5, 5, 3, 1, 4, 3, 4, 6, 3, 6, 4, 4, 6, 5, 2, 1, 2, 4, 6, 2, 5, 4, 4, 4, 5, 3, 2, 4, 2, 2, 2, 1, 4, 6, 2, 2, 5, 5, 6, 2, 1, 3, 4, 4, 2, 4, 4, 1, 4, 2, 4, 4, 3, 3, 1, 4, 6, 2, 1, 5, 6, 1, 4, 5, 6, 6, 4, 5, 4, 4, 6, 5, 4, 2, 3, 1, 4, 1, 3, 4, 1, 6, 2, 1, 3, 6, 3, 2, 6, 4, 1, 3, 4, 2, 1, 3, 3, 2, 5, 4, 2, 3, 6, 6, 5, 4, 2, 5, 5, 4, 4, 3, 3, 6, 6, 5, 5, 1, 2, 6, 1, 5, 3, 3, 5, 6, 2, 5, 6, 4, 4, 2, 4, 4, 4, 3, 6, 6, 1, 1, 3, 2, 6, 3, 2, 2, 1, 1, 6, 1, 2, 3, 4, 2, 5, 6, 6, 1, 3, 1, 5, 5, 3, 1, 1, 5, 1, 2, 4, 1, 5, 4, 1, 3, 6, 4, 3, 3, 1, 2, 4, 2, 2, 1, 2, 4, 5, 3, 4, 5, 1, 6, 5, 5, 3, 6, 3, 6, 2, 2, 2, 6, 6, 6, 5, 2, 5, 2, 5, 5, 1, 2, 3, 2, 1, 3, 2, 3, 4, 1, 6, 3, 2, 2, 2, 2, 5, 3, 2, 3, 1, 4, 4, 5, 5, 2, 2, 1, 1, 6, 2, 5, 4, 6, 5, 4, 5, 1, 5, 2, 3, 3, 2, 3, 2, 5, 1, 2, 6, 1, 2, 4, 2, 2, 3, 1, 5, 4, 6, 5, 3, 1, 4, 3, 3, 6, 3, 4, 1, 5, 3, 1, 1, 6, 2, 3, 4, 2, 6, 4, 6, 1, 5, 6, 1, 5, 4, 2, 1, 6, 3, 1, 6, 6, 2, 4, 2, 5, 1, 4, 2, 4, 4, 6, 6, 4, 5, 1, 4, 6, 2, 4, 5, 4, 4, 1, 2, 2, 6, 4, 2, 3, 1, 5, 6, 4, 1, 2, 2, 5, 4, 2, 5, 1, 1, 5, 2, 1, 4, 4, 4, 2, 1, 5, 4, 3, 5, 3, 2, 3, 5, 2, 5, 1, 2, 2, 6, 6, 5, 5, 5, 4, 3, 6, 6, 5, 1, 1, 6, 5, 3, 4, 5, 4, 1, 6, 5, 1, 5, 6, 3, 1, 5, 2, 1, 3, 4, 4, 4, 6, 5, 4, 3, 3, 3, 5, 1, 4, 5, 3, 3, 4, 2, 2, 3, 3, 5, 2, 5, 5, 2, 1, 6, 5, 4, 2, 6, 1, 6, 3, 3, 4, 6, 2, 3, 4, 4, 1, 5, 6, 5, 6, 6, 1, 2, 6, 6, 4, 4, 4, 2, 5, 1, 2, 5, 4, 2, 4, 5, 6, 6, 1, 2, 1, 1, 4, 6, 5, 4, 2, 1, 6, 5, 2, 2, 6, 6, 5, 6, 6, 6, 3, 4, 1, 2, 6, 4, 1, 2, 4, 4, 4, 3, 4, 2, 2, 6, 2, 6, 2, 1, 4, 4, 5, 1, 3, 2, 2, 4, 6, 6, 5, 3, 5, 4, 3, 3, 6, 1, 4, 2, 6, 2, 1, 4, 5, 5, 1, 4, 1, 4, 4, 4, 1, 3, 4, 1, 2, 4, 6, 5, 1, 3, 5, 1, 1, 2, 5, 6, 3, 3, 1, 2, 3, 6, 5, 1, 4, 1, 4, 2, 1, 3, 6, 2, 3, 2, 1, 5, 1, 5, 4, 1, 3, 1, 4, 3, 5, 5, 1, 6, 6, 6, 4, 1, 2, 5, 2, 2, 6, 1, 1, 1, 6, 1, 1, 1, 2, 4, 1, 2, 5, 5, 2, 6, 1, 5, 4, 2, 3, 2, 2, 4, 1, 3, 2, 1, 2, 6, 3, 3, 1, 5, 2, 1, 4, 4, 4, 3, 3, 6, 5, 6, 1, 4, 5, 2, 5, 2, 6, 1, 5, 3, 6, 1, 2, 5, 3, 3, 2, 4, 1, 5, 2, 5, 5, 5, 5, 4, 4, 2, 4, 2, 4, 4, 6, 1, 5, 4, 1, 2, 2, 5, 1, 3, 4, 3, 3, 4, 1, 5, 4, 4, 6, 6, 2, 4, 6, 2,

[damned]

3, 5, 2, 5, 4, 6, 1, 3, 6, 4, 6, 4, 2, 2, 5, 4, 5, 2, 1, 6, 3, 4, 4, 3, 2, 6, 4, 6, 6, 4, 3, 1, 2, 5, 4, 3, 5, 3, 5, 1, 1, 4, 1, 2, 3, 2, 6, 2, 2, 2, 3, 2, 6, 6, 3, 6, 2, 3, 2, 1, 6, 6, 4, 2, 6, 5, 2, 3, 4, 4, 6, 2, 3, 3, 5, 4, 6, 4, 3, 3, 1, 1, 3, 4, 3, 2, 2, 5, 3, 6, 1, 3, 4, 1, 2, 6, 4, 5, 2, 2, 5, 3, 5, 2, 1, 6, 3, 2, 1, 4, 6, 2, 2, 6, 1, 4, 3, 6, 1, 3, 1, 2, 1, 5, 6, 6, 5, 4, 4, 6, 1, 4, 2, 5, 6, 6, 5, 3, 1, 4, 4, 1, 3, 4, 2, 1, 1, 3, 4, 6, 4, 6, 4, 5, 1, 4, 3, 1, 6, 3, 6, 6, 1, 1, 4, 2, 6, 4, 2, 4, 3, 6, 6, 2, 4, 1, 3, 5, 6, 5, 3, 4, 4, 5, 4, 2, 6, 1, 5, 3, 3, 5, 3, 3, 1, 3, 1, 4, 2, 2, 3, 4, 4, 2, 2, 3, 2, 1, 3, 3, 1, 6, 2, 4, 2, 2, 2, 3, 5, 1, 5, 2, 5, 5, 6, 2, 3, 3, 4, 1, 5, 4, 3, 4, 6, 5, 3, 6, 3, 3, 4, 1, 6, 4, 4, 5, 6, 5, 6, 3, 4, 3, 2, 4, 2, 1, 2, 5, 6, 6, 3, 4, 1, 1, 3, 2, 1, 3, 4, 2, 2, 5, 3, 2, 1, 6, 1, 2, 5, 2, 5, 6, 4, 1, 5, 2, 2, 4, 3, 1, 4, 5, 6, 5, 6, 4, 6, 5, 1, 3, 1, 6, 4, 3, 6, 3, 6, 6, 4, 5, 6, 6, 5, 4, 2, 1, 4, 2, 4, 2, 1, 2, 4, 1, 5, 6, 4, 2, 6, 6, 5, 4, 3, 2, 4, 1, 1, 4, 1, 3, 3, 2, 2, 1, 6, 5, 6, 3, 1, 2, 1, 2, 4, 4, 2, 4, 5, 3, 5, 1, 6, 5, 4, 2, 3, 4, 1, 3, 6, 3, 3, 5, 3, 4, 5, 2, 2, 4, 5, 2, 2, 4, 2, 1, 6, 6, 4, 1, 1, 1, 3, 1, 5, 6, 1, 1, 6, 5, 3, 2, 1, 5, 4, 2, 2, 1, 2, 2, 2, 1, 2, 2, 4, 3, 5, 4, 2, 5, 4, 4, 2, 1, 5, 3, 2, 6, 2, 2, 2, 6, 1, 6, 5, 6, 4, 6, 3, 1, 3, 6, 2, 5, 5, 6, 6, 5, 6, 2, 5, 3, 4, 6, 6, 5, 1, 1, 5, 3, 4, 1, 4, 3, 5, 3, 4, 5, 2, 3, 5, 5, 2, 6, 1, 1, 6, 2, 6, 1, 2, 4, 2, 5, 5, 5, 5, 6, 4, 4, 5, 5, 2, 2, 5, 5, 3, 5, 2, 5, 5, 2, 2, 1, 4, 2, 6, 3, 2, 2, 3, 4, 3, 5, 2, 1, 5, 6, 1, 2, 3, 4, 2, 4, 4, 6, 3, 2, 1, 3, 4, 3, 6, 1, 2, 2, 2, 4, 1, 1, 3, 4, 4, 3, 3, 3, 4, 4, 6, 3, 2, 1, 2, 4, 5, 2, 2, 1, 2, 1, 1, 4, 6, 4, 1, 1, 5, 6, 1, 2, 2, 6, 4, 6, 3, 3, 3, 1, 4, 6, 4, 4, 5, 3, 1, 3, 2, 1, 1, 1, 2, 3, 2, 6, 1, 2, 3, 6, 3, 3, 3, 5, 2, 6, 3, 5, 1, 2, 6, 3, 5, 3, 1, 3, 5, 5, 1, 4, 6, 6, 6, 2, 2, 5, 4, 1, 3, 5, 5, 5, 2, 6, 6, 4, 2, 4, 2, 4, 5, 1, 6, 6, 3, 4, 6, 3, 2, 4, 2, 1, 2, 3, 3, 4, 2, 6, 4, 5, 3, 5, 6, 2, 2, 5, 6, 3, 3, 5, 5, 6, 3, 4, 4, 1, 5, 2, 4, 4, 6, 3, 3, 6, 5, 4, 3, 3, 4, 6, 1, 1, 2, 5, 5, 5, 3, 3, 3, 1, 4, 6, 2, 1, 5, 2, 1, 6, 6, 4, 1, 2, 2, 3, 6, 2, 2, 3, 6, 2, 1, 4, 3, 4, 6, 3, 3, 3, 3, 6, 4, 1, 6, 2, 4, 4, 3, 1, 3, 4, 6, 1, 5, 2, 3, 3, 6, 1, 3, 4, 4, 2, 6, 4, 1, 2, 3, 6, 2, 5, 2, 6, 2, 3, 5, 2, 3, 4, 4, 1, 5, 3, 5, 5, 2, 1, 5, 2, 4, 1, 1, 5, 6, 1, 5, 5, 4, 1, 5, 1, 6, 2, 2, 1, 5, 4, 4, 4, 4, 3, 3, 4, 5, 2, 5, 1, 2, 2, 4, 5, 3, 1, 1, 1, 2, 2, 3, 5, 2, 6, 4, 3, 4, 4, 3, 4, 4, 5, 2, 5, 3, 3, 6, 1, 6, 5, 4, 1, 5, 1, 3, 6, 3, 4, 3, 2, 1, 3, 2, 2, 4, 5, 5, 1, 6, 3, 3, 3, 2, 4, 3, 1, 2, 4, 4, 5, 3, 2, 5, 2, 4, 3, 4, 5, 4, 6, 6, 5, 4, 4, 5, 3, 4, 5, 2, 2, 2, 5, 4, 3, 5, 4, 6, 1, 4, 2, 4, 5, 5, 3, 3, 5, 4, 1, 4, 1, 6, 2, 2, 4, 1, 5, 6, 5, 5, 6, 1, 6, 3, 5, 3, 4, 2, 1, 3, 2, 4, 1, 4, 5, 5, 3, 3, 2, 1, 6, 3, 6, 3, 3, 3, 2, 4, 4, 6, 5, 4, 4, 4, 1, 1, 2, 1, 6, 4, 1, 5, 2, 1, 1, 3, 3, 3, 4, 4, 5, 1, 6, 1, 5, 1, 2, 6, 1, 4, 6, 6, 3, 4, 3, 5, 1, 2, 1, 6, 6, 1, 2, 6, 4, 2, 1, 1, 6, 6, 6, 1, 1, 2, 3, 4, 3, 1, 6, 6, 3, 3, 6, 5, 1, 6, 1, 2, 3, 3, 5, 6, 2, 1, 3, 1, 3, 2, 4, 5, 6, 2, 4, 2, 3, 2, 4, 2, 1, 5, 5, 6, 5, 6, 6, 3, 1, 2, 4, 2, 1, 3, 2, 6, 5, 4, 5, 4, 2, 6, 1, 5, 2, 5, 6, 1, 1, 1, 2, 6, 6, 2, 1, 6, 4, 3, 1, 4, 6, 2, 2, 3, 5, 4, 1, 6, 5, 2, 3, 1, 1, 1, 1, 2, 3, 1, 5, 5, 1, 1, 2, 1, 5, 5, 4, 3, 4, 3, 4, 3, 1, 6, 3, 3, 2, 1, 2, 1, 1, 5, 4, 4, 3, 1, 4, 1, 3, 1, 5, 6, 4, 2, 1, 5, 6, 1, 5, 2, 3, 6, 3, 4, 3, 3, 6, 4, 6, 4, 3, 2, 6, 2, 6, 6, 5, 5, 1, 2, 1, 5, 2, 2, 6, 4, 4, 4, 5, 6, 6, 6, 6, 2, 3, 1, 2, 1, 1, 3, 6, 6, 5, 1, 4, 4, 2, 3, 3, 1, 1, 2, 5, 3, 2, 1, 1, 1, 6, 1, 2, 3, 1, 4, 3, 1, 4, 3, 1, 1, 5, 6, 2, 3, 5, 2, 4, 3, 6, 1, 5, 2, 4, 6, 4, 3, 3, 1, 3, 2, 5, 3, 3, 1, 4, 3, 4, 6, 2, 3, 2, 1, 3, 3, 1, 5, 3, 4, 5, 3, 3, 6, 2, 1, 1, 5, 5, 5, 4, 6, 5, 5, 2, 3, 2, 5, 1, 3, 4, 5, 3, 1, 4, 2, 4, 6, 2, 6, 3, 3, 1, 2, 6, 1, 6, 2, 4, 3, 5, 5, 1, 1, 4, 2, 3, 5, 1, 4, 2, 3, 5, 2, 4, 2, 5, 3, 2, 6, 1, 4, 3, 6, 4, 1, 2, 4, 6, 4, 1, 2, 4, 1, 3, 4, 5, 3, 6, 3, 4, 6, 4, 4, 5, 3, 2, 2, 6, 6, 3, 3, 2, 5, 5, 5, 5, 2, 2, 6, 1, 6, 4, 1, 6, 3, 2, 3, 1, 1, 1, 3, 6, 2, 3, 4, 6, 1, 3, 1, 4, 3, 5, 2, 5, 1, 1, 1, 2, 1, 4, 1, 3, 1, 6, 2, 5, 5, 2, 3, 1, 5, 4, 5, 5, 4, 4, 1, 1, 2, 4, 1, 6, 4, 3, 4, 6, 5, 4, 3, 5, 1, 3, 3, 5, 6, 6, 1, 5, 4, 3, 6, 6, 5, 5, 6, 4, 6, 5, 4, 2, 5, 1, 1, 4, 2, 6, 5, 6, 6, 4, 4, 1, 1, 2, 5, 5, 1, 5, 6, 6, 4, 2, 3, 1, 3, 1, 1, 2, 4, 2, 5, 1, 1, 3, 5, 5, 4, 5, 5, 4, 1, 2, 1, 4, 4, 1, 1, 1, 6, 3, 3, 2, 1, 2, 5, 4, 1, 1, 3, 6, 3, 2, 3, 5, 4, 2, 4, 1, 3, 5, 1, 4, 4, 2, 3, 4, 6, 6, 4, 3, 3, 5, 2, 6, 2, 3, 4, 2, 6, 5, 5, 4, 3, 1, 5, 3, 2, 2, 5, 3, 5, 2, 5, 1, 4, 6, 3, 3, 3, 6, 3, 2, 4, 2, 1, 4, 6, 2, 2, 1, 2, 5, 5, 4, 1, 3, 6, 3, 2, 3, 2, 4, 1, 4, 6, 6, 5, 6, 4, 3, 1, 1, 5, 5, 6, 6, 2, 1, 3, 2, 4, 4, 4, 1, 4, 4, 2, 4, 4, 4, 6, 2, 4, 4, 2, 2, 4, 2, 6, 3, 4, 6, 4, 4, 6, 6, 4, 1, 3, 6, 5, 3, 5, 1, 4, 2, 4, 3, 5, 4, 1, 2, 5, 5, 1, 2, 2, 1, 6, 3, 1, 2, 6, 3, 2, 4, 2, 1, 3, 5, 5, 1, 5, 1, 2, 4, 2, 3, 2, 2, 1, 2, 3, 5, 5, 5, 6, 3, 2, 4, 6, 6, 1, 5, 3, 1, 2, 5, 1, 2, 4, 1, 3, 3, 6, 1, 6, 1, 4, 6, 6, 4, 5, 4, 4, 3, 6, 4, 4, 3, 6, 2, 2, 4, 5, 3, 6, 5, 4, 2, 2, 1, 1, 4, 6, 1, 1, 4, 6, 4, 4, 3, 2, 6, 2, 1, 6, 5, 3, 3, 4, 5, 1, 5, 3, 4, 4, 6, 1, 1, 1, 4, 4, 6, 4, 3, 4, 6, 1, 4, 5, 3, 2, 6, 4, 6, 5, 5, 3, 4, 6, 6, 4, 2, 6, 3, 1, 3, 4, 3, 5, 6, 4, 6, 4, 5, 4, 2, 5, 1, 1, 4, 3, 1, 2, 5, 1, 6, 6, 4, 2, 5, 5, 3, 3, 2, 1, 6, 6, 2, 4, 1, 2, 2, 3, 1, 3, 6, 1, 3, 6, 5, 1, 1, 5, 5, 4, 3, 4, 6, 1, 2, 4, 4, 2, 1, 6, 4, 4, 1, 1, 2, 6, 4, 4, 2, 5, 4, 2, 3, 3, 2, 1, 6, 4, 4, 1, 5, 3, 5, 1, 6, 4, 6, 6, 4, 1, 4, 2, 1, 4, 1, 2, 3, 6, 5, 6, 6, 1, 6, 6, 6, 5, 2, 6, 6, 2, 2, 2, 3, 1, 6, 6, 5, 3, 4, 1, 2, 6, 6, 2, 2, 4, 4, 1, 1, 4, 3, 4, 1, 3, 4, 5, 2, 5, 3, 2, 2, 2, 3, 4, 2, 2, 3, 1, 3, 3, 4, 1, 5, 2, 3, 3, 3, 2, 6, 6, 3, 6, 6, 3, 1, 1, 1, 1, 3, 2, 6, 6, 6, 6, 4, 1, 2, 3, 6, 3, 4, 6, 3, 5, 4, 4, 1, 6, 1, 1, 1, 6, 4, 2, 4, 2, 3, 4, 2, 3, 5, 6, 5, 3, 2, 4, 1, 3, 4, 4, 3, 3, 5, 5, 6, 4, 4, 4, 3, 1, 5, 6, 4, 3, 6, 2, 1, 3, 6, 5, 1, 3, 1, 6, 3, 5, 3, 3, 1, 5, 3, 1, 4, 4, 1, 4, 1, 2, 2, 4, 4, 4, 1, 6, 4, 1, 1, 6, 2, 5, 6, 6, 3, 4, 3, 4, 2, 4, 1, 5, 4, 3, 5, 5, 5, 1, 4, 5, 6, 1, 2, 1, 5, 4, 4, 3, 6, 1, 3, 3, 1, 1, 4, 1, 6, 1, 5, 3, 4, 5, 6, 4, 6, 6, 2, 2, 2, 3, 3, 5, 4, 1, 5, 1, 6, 3, 4, 6, 2, 6, 2, 2, 1, 4, 4, 3, 1, 3, 2, 4, 4, 1, 2, 3, 6, 2, 3, 4, 1, 2, 2, 6, 2, 3, 5, 3, 1, 2, 2, 3, 2, 3, 1, 2, 2, 2, 6, 4, 2, 1, 5, 3, 5, 6, 1, 2, 4, 1, 2, 2, 1, 2, 1, 4, 6, 2, 4, 5, 2, 6, 6, 2, 1, 3, 4, 1, 5, 4, 2, 5, 3, 6, 4, 2, 6, 1, 6, 4, 6, 6, 2, 5, 2, 3, 2, 4, 5, 3, 4, 1, 5, 5, 5, 4, 3, 2, 3, 6, 5, 1, 3, 5, 4, 4, 4, 2, 6, 6, 3, 6, 1, 4, 6, 4, 4, 1, 5, 4, 3, 4, 4, 3, 2, 4, 2, 6, 1, 4, 5, 1, 2, 1, 4, 4, 6, 4, 3, 1, 4, 1, 1, 2, 6, 1, 3, 5, 6, 2, 2, 4, 4, 1, 6, 6, 6, 2, 6, 2, 4, 3, 1, 6, 2, 5, 1, 4, 6, 1, 1, 1, 3, 2, 1, 1, 6, 6, 3, 4, 2, 4, 5, 5, 2, 3, 5, 3, 4, 6, 4, 2, 3, 5, 6, 1, 5, 3, 6, 3, 5, 3, 2, 2, 3, 2, 5, 4, 2, 1, 2, 2, 4, 5, 1, 1, 2, 6, 2, 5, 1, 1, 2, 3, 3, 5, 5, 5, 1, 5, 3, 5, 3, 6, 4, 2, 2, 5, 1, 5, 2, 4, 2, 1, 4, 5, 4, 5, 4, 3, 5, 4, 4, 4, 6, 1, 1, 5, 4, 2, 5, 1, 6, 5, 3, 5, 1, 3, 4, 5, 5, 6, 4, 5, 1, 5, 3, 3, 2, 2, 2, 1, 2, 4, 5, 6, 2, 3, 6, 6, 5, 1, 6, 1, 2, 4, 2, 1, 2, 2, 4, 6, 2, 2, 6, 4, 5, 2, 2, 2, 2, 6, 2, 3, 2, 5, 2, 1, 4, 6, 5, 2, 1, 5, 6, 5, 6, 5, 5, 3, 1, 5, 2, 4, 2, 3, 1, 3, 4, 1, 3, 6, 5, 5, 3, 6, 6, 6, 5, 6, 1, 6, 3, 3, 2, 4, 3, 4, 3, 4, 4, 6, 6, 4, 3, 6, 1, 3, 4, 6, 1, 6, 5, 5, 6, 5, 1, 6, 1, 2, 4, 3, 4, 3, 1, 3, 5, 1, 2, 2, 1, 6, 1, 1, 5, 6, 4, 4, 2, 4, 5, 3, 3, 5, 4, 4, 1, 3, 1, 1, 1, 4, 1, 5, 2, 1, 5, 6, 4, 4, 4, 5, 3, 4, 3, 5, 4, 3, 2, 1, 3, 2, 3, 4, 4, 2, 3, 2, 1, 2, 1, 4, 5, 6, 5, 6, 2, 1, 3, 3, 1, 3, 5, 3, 6, 1, 6, 5, 1, 5, 5, 1, 4, 3, 6, 5, 3, 1, 5, 6, 3, 4, 5, 4, 4, 3, 2, 2, 6, 4, 1, 3, 2, 1, 1, 2, 2, 2, 3, 3, 3, 1, 6, 3, 3, 4, 2, 3, 2, 1, 1, 2, 5, 6, 4, 2, 4, 5, 6, 6, 6, 1, 4, 4, 3, 1, 1, 2, 4, 1, 5, 1, 4, 3, 1, 2, 5, 1, 3, 5, 6, 2, 1, 4, 2, 6, 2, 4, 3, 4, 3, 2, 2, 4, 3, 2, 3, 4, 6, 5, 4, 5, 5, 3, 1, 2, 6, 3, 5, 5, 1, 6, 4, 5, 4, 6, 1, 3, 6, 4, 1, 6, 1, 4, 4, 5, 2, 5, 6, 1, 3, 3, 6, 4, 1, 6, 5, 3, 3, 3, 2, 6, 3, 5, 2, 1, 3, 6, 3, 6, 3, 6, 1, 6, 2, 6, 6, 6, 3, 3, 2, 5, 6, 4, 2, 1, 4, 3, 2, 6, 4, 3, 1, 1, 6, 3, 1, 1, 6, 1, 1, 2, 1, 4, 6, 4, 6, 1, 4, 1, 5, 2, 2, 4, 4, 6, 5, 2, 6, 4, 2, 2, 2, 2, 6, 6, 4, 2, 4, 5, 3, 5, 1, 1, 3, 5, 4, 4, 2, 3, 6, 6, 6, 4, 2, 3, 4, 5, 3, 6, 4, 4, 2, 2, 1, 1, 3, 3, 6, 5, 3, 3, 4, 3, 6, 5, 3, 5, 1, 4, 3, 3, 1, 5, 4, 5, 6, 6, 1, 3, 5, 4, 6, 4, 4, 4, 4, 3, 5, 6, 2, 6, 3, 4, 2, 3, 4, 6, 1, 4, 6, 6, 4, 2, 5, 1, 6, 4, 3, 4, 2, 4, 6, 4, 4, 1, 1, 3, 5, 6, 5, 5, 4, 3, 5, 5, 1, 2, 3, 3, 4, 5, 6, 6, 2, 5, 3, 6, 2, 5, 1, 6, 6, 2, 3, 3, 4, 2, 5, 1, 1, 5, 4, 4, 2, 1, 5, 5, 4, 4, 6, 1, 5, 5, 2, 3, 4, 5, 5, 2, 1, 4, 6, 4, 6, 1, 2, 3, 4, 2, 1, 3, 4, 6, 1, 1, 1, 3, 2, 2, 4, 1, 2, 4, 2, 6, 1, 1, 2, 2, 3, 4, 2, 2, 5, 1, 2, 6, 1, 2, 4, 2, 1, 4, 1, 6, 3, 3, 5, 1, 1, 6, 1, 2, 2, 4, 1, 1, 4, 6, 2, 4, 2, 6, 6, 3, 4, 3, 4, 3, 1, 4, 3, 4, 4, 6, 3, 4, 5, 6, 4, 5, 5, 2, 6, 2, 4, 2, 2, 6, 2, 1, 2, 1, 1, 2, 1, 3, 5, 1, 1, 1, 3, 3, 5, 4, 3, 5, 1, 4, 1, 4, 2, 5, 4, 2, 4, 1, 4, 4, 4, 2, 2, 5, 1, 1, 6, 6, 4, 2, 6, 4, 6, 4, 4, 3, 6, 2, 4, 1, 6, 3, 6, 3, 5, 6, 5, 1, 1, 1, 5, 6, 2, 5, 3, 2, 2, 4, 1, 3, 2, 2, 5, 4, 2, 2, 6, 3, 2, 4, 6, 1, 6, 5, 1, 2, 2, 4, 6, 2, 1, 5, 5, 2, 5, 5, 2, 4, 4, 5, 4, 1, 5, 6, 3, 5, 6, 2, 1, 4, 6, 1, 3, 2, 4, 3, 1, 6, 1, 4, 5, 4, 6, 1, 1, 1, 6, 5, 3, 4, 2, 4, 4, 4,

[damned]

3, 1, 1, 4, 1, 3, 3, 4, 3, 2, 4, 5, 3, 5, 5, 3, 4, 4, 4, 6, 5, 6, 2, 1, 6, 2, 3, 2, 4, 2, 2, 5, 4, 1, 2, 6, 2, 5, 6, 2, 6, 1, 5, 5, 2, 4, 2, 2, 2, 4, 5, 1, 1, 5, 2, 1, 1, 6, 2, 4, 5, 2, 6, 5, 2, 4, 6, 6, 1, 5, 4, 3, 3, 6, 6, 2, 5, 6, 3, 1, 3, 5, 6, 4, 2, 6, 2, 2, 2, 2, 4, 5, 1, 2, 5, 3, 1, 3, 4, 3, 3, 2, 6, 2, 5, 6, 3, 4, 2, 6, 6, 4, 3, 2, 2, 4, 5, 4, 3, 6, 3, 2, 5, 1, 1, 3, 6, 5, 2, 2, 6, 5, 6, 5, 1, 4, 2, 5, 3, 4, 5, 5, 5, 3, 3, 6, 4, 5, 6, 3, 6, 1, 6, 3, 2, 3, 1, 1, 4, 2, 2, 6, 2, 4, 6, 1, 3, 1, 2, 4, 2, 6, 2, 1, 6, 1, 4, 2, 3, 5, 5, 5, 6, 3, 3, 6, 4, 6, 3, 4, 1, 6, 1, 3, 4, 3, 5, 3, 4, 6, 6, 2, 6, 1, 5, 1, 6, 5, 1, 4, 1, 5, 5, 1, 2, 1, 1, 3, 1, 4, 4, 6, 5, 3, 2, 4, 5, 4, 5, 5, 4, 1, 2, 3, 3, 2, 3, 5, 6, 4, 1, 5, 1, 1, 4, 1, 1, 4, 2, 1, 3, 5, 4, 4, 3, 5, 4, 1, 5, 6, 2, 4, 6, 2, 2, 6, 1, 2, 6, 5, 6, 6, 5, 3, 2, 2, 1, 4, 4, 4, 5, 4, 3, 2, 6, 2, 5, 2, 4, 3, 2, 5, 3, 1, 3, 4, 1, 6, 6, 6, 1, 3, 6, 1, 2, 3, 5, 3, 3, 1, 2, 3, 5, 3, 6, 1, 2, 6, 3, 4, 6, 3, 5, 4, 4, 2, 5, 2, 4, 1, 2, 2, 2, 5, 6, 1, 6, 4, 5, 5, 4, 6, 6, 2, 6, 1, 3, 1, 1, 2, 2, 4, 2, 2, 4, 3, 5, 2, 1, 6, 3, 2, 6, 3, 1, 2, 1, 3, 3, 3, 2, 6, 4, 6, 6, 5, 1, 1, 2, 6, 5, 6, 6, 4, 6, 2, 4, 4, 6, 3, 2, 6, 4, 5, 6, 3, 4, 4, 5, 5, 2, 5, 3, 4, 3, 2, 6, 5, 1, 6, 3, 1, 2, 1, 4, 5, 1, 5, 6, 2, 5, 4, 6, 4, 6, 6, 3, 3, 6, 2, 1, 6, 2, 2, 2, 1, 3, 3, 4, 2, 4, 5, 1, 3, 3, 2, 5, 1, 1, 2, 2, 6, 6, 1, 6, 1, 4, 1, 4, 6, 4, 4, 5, 3, 1, 3, 3, 2, 1, 1, 2, 6, 3, 2, 4, 2, 2, 2, 5, 2, 6, 1, 4, 3, 4, 2, 6, 6, 1, 3, 6, 2, 5, 4, 1, 1, 5, 6, 5, 5, 1, 6, 1, 3, 2, 5, 3, 4, 4, 1, 3, 1, 1, 1, 3, 6, 1, 3, 4, 6, 4, 6, 3, 1, 3, 3, 3, 4, 3, 1, 3, 2, 2, 3, 1, 1, 1, 1, 5, 6, 3, 1, 4, 1, 1, 6, 2, 4, 1, 5, 6, 4, 1, 1, 6, 2, 2, 3, 4, 5, 4, 5, 6, 5, 3, 2, 1, 2, 1, 3, 5, 5, 5, 4, 4, 1, 2, 1, 1, 3, 4, 5, 6, 6, 5, 3, 6, 6, 6, 5, 1, 1, 1, 6, 1, 5, 4, 4, 4, 3, 4, 5, 6, 3, 5, 6, 4, 2, 3, 5, 6, 2, 5, 3, 4, 4, 5, 2, 3, 5, 1, 3, 3, 4, 2, 3, 2, 1, 2, 6, 4, 6, 2, 5, 6, 6, 6, 4, 6, 5, 6, 3, 6, 3, 4, 6, 1, 1, 2, 4, 5, 3, 4, 6, 2, 2, 6, 2, 6, 3, 2, 3, 3, 6, 2, 2, 2, 3, 1, 1, 2, 3, 5, 6, 5, 4, 5, 4, 6, 2, 2, 4, 4, 1, 3, 3, 1, 5, 3, 2, 6, 2, 2, 3, 4, 6, 1, 6, 6, 2, 2, 1, 3, 1, 1, 3, 6, 3, 5, 2, 2, 2, 4, 3, 4, 4, 1, 2, 6, 5, 1, 2, 6, 4, 3, 1, 5, 1, 5, 4, 1, 5, 1, 1, 3, 4, 3, 5, 3, 4, 6, 6, 3, 3, 4, 4, 4, 6, 5, 2, 3, 1, 4, 1, 1, 4, 1, 1, 2, 2, 2, 4, 1, 3, 3, 2, 5, 3, 5, 2, 1, 1, 4, 1, 6, 4, 2, 2, 6, 1, 6, 6, 4, 6, 4, 3, 1, 5, 3, 6, 4, 2, 4, 6, 1, 3, 4, 2, 3, 6, 2, 2, 5, 5, 2, 2, 5, 5, 4, 3, 2, 5, 5, 4, 1, 4, 1, 3, 3, 2, 3, 3, 5, 6, 4, 1, 5, 6, 5, 3, 3, 3, 4, 2, 3, 2, 4, 4, 4, 6, 1, 1, 6, 5, 1, 5, 1, 6, 5, 2, 2, 3, 6, 4, 4, 3, 2, 4, 3, 4, 1, 4, 3, 3, 6, 4, 4, 3, 4, 5, 5, 5, 2, 4, 6, 3, 4, 3, 4, 6, 1, 3, 3, 4, 1, 3, 3, 4, 5, 4, 3, 2, 2, 4, 4, 1, 1, 1, 4, 4, 5, 3, 3, 6, 5, 1, 4, 4, 2, 1, 2, 5, 4, 6, 4, 3, 1, 6, 3, 1, 1, 5, 5, 3, 2, 4, 6, 6, 5, 1, 2, 2, 6, 6, 2, 3, 6, 2, 1, 2, 5, 2, 2, 1, 4, 3, 3, 5, 3, 2, 1, 3, 1, 5, 1, 1, 4, 5, 3, 1, 2, 2, 5, 2, 4, 1, 2, 2, 3, 4, 6, 6, 1, 2, 6, 5, 5, 4, 3, 2, 1, 4, 3, 3, 4, 5, 3, 1, 5, 1, 4, 5, 4, 1, 2, 2, 2, 2, 4, 2, 6, 4, 2, 3, 5, 1, 3, 3, 2, 3, 2, 6, 4, 2, 3, 2, 4, 4, 3, 4, 2, 2, 1, 3, 1, 6, 5, 2, 3, 4, 5, 2, 5, 6, 6, 5, 2, 4, 1, 1, 5, 5, 5, 4, 2, 5, 1, 6, 5, 3, 6, 2, 2, 2, 6, 6, 1, 3, 4, 6, 3, 4, 4, 4, 2, 1, 2, 4, 3, 2, 5, 2, 1, 6, 1, 1, 3, 3, 4, 2, 6, 5, 3, 4, 2, 6, 3, 2, 6, 3, 1, 5, 5, 6, 1, 2, 4, 4, 6, 1, 4, 4, 4, 1, 4, 2, 1, 6, 4, 4, 6, 5, 4, 2, 2, 5, 1, 6, 1, 2, 3, 1, 1, 6, 6, 6, 2, 3, 2, 1, 1, 3, 5, 4, 2, 4, 4, 2, 1, 2, 2, 2, 4, 2, 1, 4, 2, 3, 3, 3, 4, 1, 4, 3, 2, 3, 3, 1, 6, 1, 3, 6, 1, 4, 2, 3, 3, 3, 3, 2, 5, 3, 4, 3, 2, 6, 3, 5, 1, 5, 4, 2, 3, 4, 3, 5, 5, 3, 2, 5, 5, 5, 6, 1, 2, 1, 2, 6, 1, 1, 1, 1, 5, 6, 6, 1, 3, 6, 4, 5, 6, 3, 4, 5, 3, 6, 5, 6, 1, 2, 5, 5, 2, 4, 5, 6, 2, 1, 2, 1, 3, 2, 4, 2, 3, 3, 2, 6, 4, 3, 4, 6, 4, 6, 4, 6, 5, 5, 1, 6, 4, 2, 3, 1, 2, 3, 5, 3, 4, 1, 4, 5, 2, 3, 5, 6, 1, 2, 2, 6, 6, 6, 2, 3, 6, 6, 6, 4, 6, 2, 4, 3, 4, 1, 2, 3, 4, 4, 5, 3, 4, 2, 5, 2, 3, 6, 3, 1, 4, 2, 6, 3, 6, 5, 2, 3, 5, 5, 3, 4, 6, 6, 4, 5, 5, 6, 5, 6, 5, 3, 2, 6, 2, 3, 5, 2, 1, 4, 1, 2, 3, 3, 5, 4, 2, 6, 2, 4, 2, 4, 2, 6, 5, 1, 6, 6, 2, 6, 4, 4, 1, 4, 6, 2, 4, 3, 3, 1, 4, 5, 4, 5, 6, 2, 2, 1, 2, 1, 1, 6, 6, 5, 2, 2, 4, 3, 6, 6, 4, 1, 3, 6, 6, 1, 2, 5, 5, 2, 5, 3, 4, 5, 2, 4, 2, 4, 1, 1, 3, 6, 5, 3, 4, 6, 4, 5, 3, 2, 1, 2, 4, 4, 4, 5, 5, 2, 4, 2, 3, 2, 1, 2, 5, 6, 1, 3, 6, 5, 1, 6, 3, 2, 3, 6, 5, 1, 1, 2, 2, 3, 1, 4, 6, 2, 6, 1, 4, 3, 5, 2, 4, 6, 6, 5, 6, 1, 2, 6, 4, 6, 1, 6, 4, 3, 1, 3, 1, 4, 3, 2, 5, 3, 1, 2, 3, 5, 1, 5, 4, 1, 4, 1, 6, 5, 6, 1, 5, 6, 5, 4, 3, 3, 1, 3, 1, 5, 4, 2, 5, 2, 6, 4, 2, 4, 2, 1, 4, 3, 6, 1, 6, 1, 5, 3, 5, 6, 2, 2, 4, 3, 3, 3, 1, 3, 4, 4, 3, 5, 5, 1, 6, 4, 4, 5, 1, 3, 6, 1, 3, 2, 2, 3, 5, 5, 5, 4, 5, 5, 5, 4, 5, 2, 4, 2, 5, 3, 6, 3, 5, 2, 2, 1, 3, 2, 1, 1, 5, 5, 2, 1, 2, 1, 4, 2, 1, 4, 5, 6, 1, 3, 2, 4, 6, 1, 5, 1, 1, 5, 2, 2, 4, 6, 5, 4, 1, 3, 2, 2, 2, 5, 4, 6, 6, 2, 1, 1, 3, 6, 1, 2, 4, 6, 3, 2, 5, 2, 5, 1, 4, 1, 2, 3, 1, 6, 1, 6, 2, 3, 6, 1, 6, 6, 1, 3, 5, 1, 4, 2, 3, 2, 6, 6, 2, 4, 6, 5, 5, 3, 1, 5, 2, 3, 3, 6, 6, 4, 5, 3, 2, 3, 4, 4, 4, 4, 3, 4, 2, 3, 4, 5, 3, 6, 4, 2, 5, 6, 4, 2, 3, 2, 1, 5, 3, 2, 4, 3, 1, 5, 5, 6, 6, 4, 3, 4, 5, 5, 6, 4, 4, 2, 3, 2, 1, 4, 2, 2, 5, 4, 1, 2, 4, 2, 5, 6, 2, 3, 3, 5, 1, 4, 3, 5, 2, 2, 5, 2, 4, 5, 4, 5, 1, 5, 1, 1, 4, 1, 6, 2, 6, 3, 3, 2, 2, 1, 3, 6, 5, 5, 2, 2, 2, 4, 1, 4, 3, 6, 1, 6, 2, 1, 3, 1, 1, 1, 5, 2, 4, 6, 5, 6, 4, 6, 4, 2, 1, 2, 4, 3, 3, 3, 3, 6, 2, 1, 1, 1, 6, 1, 3, 4, 6, 1, 1, 2, 3, 1, 2, 4, 4, 1, 6, 5, 4, 3, 4, 3, 5, 4, 4, 2, 6, 2, 5, 3, 3, 6, 6, 1, 6, 3, 4, 3, 6, 1, 4, 1, 3, 3, 1, 1, 3, 3, 6, 4, 3, 1, 5, 2, 2, 5, 4, 4, 3, 1, 3, 5, 5, 1, 6, 2, 5, 6, 1, 2, 2, 1, 6, 1, 4, 1, 1, 1, 5, 5, 5, 1, 4, 5, 2, 3, 6, 1, 6, 6, 2, 2, 3, 3, 4, 2, 3, 4, 1, 5, 1, 3, 3, 3, 6, 2, 4, 3, 5, 1, 4, 5, 1, 2, 4, 3, 6, 6, 5, 3, 1, 2, 1, 2, 2, 5, 3, 1, 6, 6, 4, 5, 1, 3, 5, 3, 5, 4, 2, 5, 4, 3, 1, 4, 3, 4, 5, 4, 6, 1, 5, 1, 4, 6, 4, 1, 1, 2, 4, 1, 6, 4, 5, 1, 4, 3, 1, 1, 1, 4, 2, 4, 3, 1, 3, 1, 4, 6, 2, 2, 3, 4, 3, 5, 6, 1, 5, 5, 6, 2, 2, 2, 6, 3, 3, 2, 6, 4, 5, 1, 3, 3, 5, 2, 1, 4, 4, 6, 5, 4, 5, 4, 1, 4, 4, 1, 1, 4, 6, 1, 3, 4, 2, 3, 3, 5, 3, 4, 4, 1, 6, 6, 1, 2, 6, 2, 5, 4, 4, 4, 2, 3, 1, 1, 2, 3, 5, 3, 1, 5, 4, 5, 3, 4, 2, 3, 2, 5, 3, 3, 6, 6, 6, 5, 1, 4, 3, 4, 1, 3, 2, 1, 6, 4, 5, 2, 5, 4, 3, 3, 5, 5, 5, 6, 6, 1, 5, 6, 1, 3, 5, 3, 3, 4, 2, 1, 4, 2, 1, 5, 2, 2, 6, 5, 1, 3, 1, 2, 4, 5, 3, 6, 5, 5, 4, 2, 2, 2, 6, 4, 2, 1, 3, 6, 4, 3, 3, 4, 4, 3, 3, 4, 5, 4, 5, 6, 3, 6, 2, 4, 3, 3, 4, 3, 6, 5, 6, 3, 6, 4, 3, 6, 3, 3, 5, 5, 4, 4, 5, 4, 2, 5, 6, 2, 2, 2, 3, 4, 2, 5, 3, 2, 2, 6, 2, 5, 4, 1, 5, 5, 1, 2, 5, 5, 6, 3, 3, 4, 5, 5, 3, 4, 3, 2, 6, 1, 2, 1, 4, 6, 4, 5, 1, 3, 3, 3, 3, 4, 1, 3, 3, 2, 2, 1, 2, 2, 3, 4, 6, 2, 2, 2, 4, 3, 3, 5, 2, 5, 1, 4, 4, 5, 6, 3, 3, 3, 6, 4, 2, 2, 1, 2, 1, 1, 2, 6, 4, 3, 3, 2, 6, 6, 6, 2, 3, 3, 6, 4, 6, 3, 2, 2, 1, 4, 1, 1, 3, 1, 5, 2, 2, 2, 1, 6, 4, 1, 6, 6, 2, 5, 2, 1, 3, 2, 4, 3, 6, 3, 2, 1, 6, 2, 2, 1, 1, 4, 4, 2, 5, 1, 2, 5, 2, 2, 4, 5, 3, 4, 6, 1, 2, 2, 5, 1, 1, 3, 6, 5, 6, 3, 2, 1, 2, 4, 4, 1, 1, 1, 5, 2, 6, 3, 4, 4, 5, 4, 5, 2, 4, 6, 4, 5, 4, 4, 5, 1, 3, 4, 2, 3, 2, 5, 3, 4, 4, 2, 6, 6, 5, 1, 1, 1, 4, 4, 5, 3, 5, 6, 5, 4, 6, 6, 1, 1, 2, 4, 5, 1, 1, 5, 6, 5, 5, 5, 5, 4, 3, 3, 5, 6, 6, 1, 2, 3, 6, 3, 2, 3, 4, 4, 3, 5, 3, 4, 2, 1, 4, 6, 2, 2, 5, 3, 2, 1, 5, 5, 4, 4, 3, 5, 3, 4, 4, 1, 1, 5, 3, 6, 1, 1, 2, 2, 5, 3, 2, 5, 2, 5, 6, 4, 4, 2, 3, 1, 3, 6, 1, 1, 2, 5, 2, 2, 5, 3, 1, 6, 1, 4, 4, 3, 3, 5, 4, 2, 2, 3, 6, 2, 1, 2, 3, 1, 5, 5, 1, 2, 4, 5, 4, 6, 2, 4, 4, 4, 4, 6, 1, 4, 5, 2, 4, 5, 2, 5, 3, 2, 5, 3, 6, 5, 4, 2, 2, 5, 1, 1, 4, 1, 3, 1, 2, 5, 3, 6, 4, 4, 1, 1, 6, 1, 5, 3, 2, 6, 3, 1, 4, 4, 1, 1, 5, 4, 4, 6, 5, 4, 1, 3, 3, 2, 3, 6, 6, 5, 3, 2, 2, 4, 6, 1, 3, 1, 6, 1, 2, 3, 1, 1, 2, 6, 4, 4, 1, 6, 5, 1, 6, 1, 4, 1, 6, 3, 3, 6, 1, 5, 5, 6, 3, 2, 4, 2, 1, 2, 4, 1, 2, 6, 6, 1, 3, 3, 4, 6, 3, 5, 4, 6, 6, 2, 1, 2, 1'

[damned]

and here is one more drop of 10,000

[seycyrus]

If you scroll up and down, again and again, you can see the woman in the red dress.

[damned]

If you scroll up and down, again and again, you can see the woman in the red dress.

:) I saw something different!

[Weissrolf]

Both of these are only 9981 rolls with a mean of 3.45 (instead of 3.5) and 2D6 mean of 6.90 (instead of 7).

I calculated the number of doubles and specific doubles for roll2.txt. On average there should be 1663 total doubles if we roll both 1+2 and 2+3 as doubles (9980 rolls), there should be 277 of each specific double then.

Doubles: 1746

1s: 295
2s: 326
3s: 282
4s: 328
5s: 237
6s: 278

At least for the sample size of 9981 rolls the means don't match the expected values. So either we need an even larger sample-size or the rolls are not as random as we would like them to be (or your extension is bugged :P).

[Weissrolf]

These are the results for the first rolls (copy & pasted from the thread). Again these were only 9981 rolls, no idea what went wrong there?! And again the mean of all rolls is only 3.45 (6.9 for 2D6) instead of 3.5 (7).

Doubles: 1715 (1663,33)

1s: 300 (277,22)
2s: 316 (277,22)
3s: 272 (277,22)
4s: 336 (277,22)
5s: 243 (277,22)
6s: 248 (277,22)

[Weissrolf]

Both rolls combined (=19960 2D6 rolls). 1D6 mean 3.45, 2D6 mean 6.9.

Doubles: 3461 (3326,67)

1s: 595 (554,44)
2s: 642 (554,44)
3s: 554 (554,44)
4s: 664 (554,44)
5s: 480 (554,44)
6s: 526 (554,44)

Now someone needs to explain to me why the mean of 19960 D6 rolls turns out to be 3.45 instead of 3.5 in FG (did you use Classic or Unity)?!

[damned]

So firstly - my roll is/was bugged. It was capturing all rolls in the tally but in the running record of rolls it was capturing roll #1 and then missing the next 19 rolls and capturing the rest due to a mistake in the code.

As to the results not being exactly as expected - I have to 100% disagree with your conclusion.
A truly randomised set of results would be expected to be close to and not actually be exactly what the mathematical averages would be.
If you rolled 10,000 times and got an exact average of 3.5 and got exactly 277 of each double then the results are being massaged to fit the theory.
If you got somewhere in the ball park and your results were not replicable over a large number of rolls then your randomness is pretty good.

The script now takes the following:

/rngroll d#x#

It will roll the specified (d#) die in lots of 20 (to speed up the process) and will roll a total number of dice as specified (x#) (use multiples of 20).
it will report
Total Number of Dice
Sum of all Dice
Average of the Sum
Frequency each face is rolled
Running list of rolls

Example

#1 Rolled: 485 Occurrence: 4%
#2 Rolled: 488 Occurrence: 4%
#3 Rolled: 524 Occurrence: 5%
#4 Rolled: 450 Occurrence: 4%
#5 Rolled: 496 Occurrence: 4%
#6 Rolled: 509 Occurrence: 5%
#7 Rolled: 529 Occurrence: 5%
#8 Rolled: 506 Occurrence: 5%
#9 Rolled: 441 Occurrence: 4%
#10 Rolled: 514 Occurrence: 5%
#11 Rolled: 509 Occurrence: 5%
#12 Rolled: 536 Occurrence: 5%
#13 Rolled: 523 Occurrence: 5%
#14 Rolled: 488 Occurrence: 4%
#15 Rolled: 476 Occurrence: 4%
#16 Rolled: 495 Occurrence: 4%
#17 Rolled: 486 Occurrence: 4%
#18 Rolled: 449 Occurrence: 4%
#19 Rolled: 564 Occurrence: 5%
#20 Rolled: 532 Occurrence: 5%
Total Number of Rolls: 10000
Sum of all Rolls: 105649
Average of all Rolls: 10.5649
Actual Rolls in Order:
6, 7, 7, 10, 13, 9, 18, 14, 13, 13, 5, 7, 18, 16, 17, 6, 12, 3, 1, 20, 17, 17, 19, 19, 13, 14, 19, 18, 20, 1, 20, 2, 7, 7, 13, 17, 7, 16, 17, 2, 12, 20, 16, 17, ....

it takes up to 15mins to make the 10,000 rolls depending on the dice type.

[damned]

d6 test

#1 Rolled: 1617 Occurrence: 16%
#2 Rolled: 1736 Occurrence: 17%
#3 Rolled: 1698 Occurrence: 16%
#4 Rolled: 1775 Occurrence: 17%
#5 Rolled: 1560 Occurrence: 15%
#6 Rolled: 1614 Occurrence: 16%
Total Number of Rolls: 10000
Sum of all Rolls: 34767
Average of all Rolls: 3.4767
Actual Rolls in Order:
6, 3, 3, 1, 1, 1, 3, 4, 1, 1, 3, 3, 5, 2, 6, 2, 2, 1, 3, 4, 6, 3, 2, 3, 5, 6, 3, 6, 1, 3, 6, 4, 1, 6, 2, 3, 6, 6, 3, 4, 5, 1, 1, 4, 3, 1, 5, 2, 5, 5, 4, 4, 6, 4,...

[damned]

d12 test

#1 Rolled: 764 Occurrence: 7%
#2 Rolled: 878 Occurrence: 8%
#3 Rolled: 845 Occurrence: 8%
#4 Rolled: 962 Occurrence: 9%
#5 Rolled: 812 Occurrence: 8%
#6 Rolled: 744 Occurrence: 7%
#7 Rolled: 876 Occurrence: 8%
#8 Rolled: 823 Occurrence: 8%
#9 Rolled: 837 Occurrence: 8%
#10 Rolled: 826 Occurrence: 8%
#11 Rolled: 745 Occurrence: 7%
#12 Rolled: 888 Occurrence: 8%
Total Number of Rolls: 10000
Sum of all Rolls: 64787
Average of all Rolls: 6.4787
Actual Rolls in Order:
5, 6, 12, 4, 5, 12, 11, 4, 11, 2, 9, 9, 7, 8, 9, 11, 10, 5, 8, 8, 11, 6, 12, 7, 5, 9, 11, 11, 9, 3, 8, 2, 10, 5, 5, 4, 12, 4, 3, 2, 7, 4, 9, 3, 9, 10, 12, 5, 7, ...

[Weissrolf]

So firstly - my roll is/was bugged. It was capturing all rolls in the tally but in the running record of rolls it was capturing roll #1 and then missing the next 19 rolls and capturing the rest due to a mistake in the code.
Obviously, but those 19 rolls don't make or break anything if the other results are still reliable.


As to the results not being exactly as expected - I have to 100% disagree with your conclusion.
So you disagree that we either need a larger data-set to check if FG's result may not be as random as we would like them to be? And that (2 times repeated) deviation seems quite high?


A truly randomised set of results would be expected to be close to and not actually be exactly what the mathematical averages would be.
Based on the current dataset of 19962 rolls an average of 3.45 seems a bit far off. Getting this exact same (too far off) average two times in a row is possible, but again an indication that we either need more rolls to analyze or something is off.


If you rolled 10,000 times and got an exact average of 3.5 and got exactly 277 of each double then the results are being massaged to fit the theory.
Which is why I wrote that we need more rolls to average if 10k is not enough to reach the projected means.


If you got somewhere in the ball park and your results were not replicable over a large number of rolls then your randomness is pretty good.
Here are average results from random.org, doing 10x 10000 integer between 1 to 6 (D6):

3,4735
3,4889
3,4962
3,5271
3,4894
3,5088
3,5211
3,4876
3,5101
3,4664

Here are the average of two 10k rolls summed up to 20k:

3,4812
3,49255
3,51165
3,50825
3,4991
3,51495
3,50435
3,49885
3,48825
3,46995

Average of all 10x10k = 100.000 rolls: 3,49691

FG averages for 9980 rolls: 3,45471 and 3,45060
FG average for 19960 rolls: 3,45266

All of these results are closer to 3.50 than what we have seen from FG yet (Classic or Unity?) and thus we need more numbers for better comparison and interpretation.

Random.org results for 9800x 2D6 rolls: average 6,96253 (FG: 6,91)

Doubles: 1722 (FG: 1715)

1s: 286 (FG: 300)
2s: 274 (FG: 316)
3s: 313 (FG: 272)
4s: 291 (FG: 336)
5s: 259 (FG: 243)
6s: 299 (FG: 248)

FG's average deviation of specific doubles compared to the perfect 277.22 is ~31.5 after 9980 rolls, random.org's average deviation is only ~16.9.

Anyway, I would prefer to compare 100k rolls instead of just 10k rolls, both for comparison the totals and for comparing the deviation of 10k packets.

Here are the random.org results for total doubles and specific doubles after 100k rolls: average 6,96253

Doubles: 16583 (16666,67)

1s: 2708 (2777,78)
2s: 2757 (2777,78)
3s: 2821 (2777,78)
4s: 2763 (2777,78)
5s: 2725 (2777,78)
6s: 2809 (2777,78)

100k rolls on random.org get us pretty close to projected means and chances. I am quite surprised that even more rolls seem to be needed to lower deviation, though.


it takes up to 15mins to make the 10,000 rolls depending on the dice type.
Both the physical rolling and the (single-thread) CPU bottleneck take time. On random.org it takes less than 1 second.

[seycyrus]

...
Which is why I wrote that we need more rolls to average if 10k is not enough to reach the projected means.

As a holder of an advanced degree, I've had the opportunity to talk to several holders of advanced degrees in statistics and probability and I can say with a fair degree of certainty that YOU should be able to determine how many rolls are needed to reach the projected means within a given window.

[Weissrolf]

No advanced degree here, just curiosity and trying to participate in a thread called "Campaign Game Statistics".

Judging from just the rolls we got yet FG's deviation seems larger than random.org for the same amount of rolls. That being said, random.org presents itself as kind of a gold standard and makes money providing random numbers, so they should be better than your average computer RNG.

[damned]

Here are two sets of 100k rolls for you and they are in my opinion both well within expected standard deviations... as were the 10k ones...

d6
#1 Rolled: 16289 Occurrence: 16%
#2 Rolled: 17863 Occurrence: 17%
#3 Rolled: 16339 Occurrence: 16%
#4 Rolled: 17444 Occurrence: 17%
#5 Rolled: 15691 Occurrence: 15%
#6 Rolled: 16374 Occurrence: 16%
Total Number of Rolls: 100000
Sum of all Rolls: 347507
Average of all Rolls: 3.47507
Actual Rolls in Order:
2, 4, 5, 6, 3, 3, 4, 5, 1, 5, 5, 2, 6, 3, 3, 2, 3, 4, 3, 3, 5, 1, 4, 6, 5, 3, 5, 3, 3, 2, 5, 1, 6, 3, 4, 3, 2, 5, 1, 1, 4, 5, 5, 2, 1, 5, 5, 5, 1, 3, 5, 6, 1, 5, ...

d10

#1 Rolled: 9996 Occurrence: 9%
#2 Rolled: 9982 Occurrence: 9%
#3 Rolled: 9647 Occurrence: 9%
#4 Rolled: 10039 Occurrence: 10%
#5 Rolled: 10236 Occurrence: 10%
#6 Rolled: 10008 Occurrence: 10%
#7 Rolled: 10082 Occurrence: 10%
#8 Rolled: 10064 Occurrence: 10%
#9 Rolled: 10129 Occurrence: 10%
#10 Rolled: 9817 Occurrence: 9%
Total Number of Rolls: 100000
Sum of all Rolls: 550702
Average of all Rolls: 5.50702
Actual Rolls in Order:
1, 2, 6, 5, 4, 8, 3, 8, 8, 1, 5, 5, 10, 1, 7, 1, 4, 4, 1, 6, 9, 9, 3, 5, 8, 7, 5, 8, 5, 3, 3, 3, 6, 7, 1, 5, 2, 1, 3, 3, 7, 4, 7, 2, 5, 8, 1, 6, 8, 7, 4, 7, 4, 10, ...

[Weissrolf]

Actually this is not "for me", but thanks for the work of providing those rolls. As soon as I find time (after sleep, work and family) I will put those into Excel and do calculations compared to random.org.

[seycyrus]

Actually this is not "for me", but thanks for the work of providing those rolls. As soon as I find time (after sleep, work and family) I will put those into Excel and do calculations compared to random.org.

So, who is it for?

I thought you were participating in this thread out of "curiosity"?

[Weissrolf]

Last time I checked this was a community forum. And my players also keep mentioning how computer generated random numbers are often flawed. So I do my best to add to the discussion from an critical point of view and it's quite a lot of work just for getting some numbers. I also do like to analyze and test software and hardware instead of relying on specs how thing "should work", though, so my own interests are met as well. ;)

[damned]

lets stay friendly people

[Weissrolf]

Well, I do have to wonder how opposed some replies sound to the idea of doing statistics in a thread about statistics. It's not like this is my favorite pastime, but sometimes things need to be clarified by hard numbers.

[seycyrus]

It's amazing to me how some people who like to ask questions, don't like it when they are the ones being asked the questions. I'm just one member of the community as well - No affiliation with SW or the forum.

The phrase "it's not for me", just struck me as an odd one, especially since it followed a comment about curiosity being the motivation.

As to your players mentioning that computer generated random numbers being flawed. Are they more flawed than "normal" dice with "normal" defects?

How do these flaws manifest in a typical game? Are we talking about something akin to an extra "20" occurring every 30 years of game (real) time? It seems unimportant, even artificial when one considers that these rolls are used to generate results for different players and NPCs, ranging from combat results, other types of skill rolls and stuff like whether or not the barmaid thinks the wizard is cute. One must also consider the players that are just rolling dice off on the side to see them fly.

I'm curious myself, but my curiosity is in the direction of whether or not complete randomness is natural and/or neccesary.

[damned]

lies, damned lies and then there is statistics.
you could roll a million times and i would not expect the average to be 3.5
likewise i would not be surprised if it did equal 3.5

"lets stay friendly" was directed at everyone

[Weissrolf]

As far as I can tell I answered all your past questions. Feel free to point me to one you are still awaiting an answer for. And the "not for me" phrase should not come as such a big surprise. Both damned and myself are dissecting FG's random number generator results in order to find evidence of how good or bad it works.

I am not sure if damned is trying to prove how good it is working against opposing views, but it seems like that. Personally I am just interested in quantitative and qualitative interpretation of real results, both to gain a personal understanding of the strengths and weaknesses of a system I paid for and work with. Sharing the results and my own interpretations is part of the community process, especially when I benefit from damned's work and like others to benefit from mine (and work it is).

Being inquisitive and critical in all directions is the basis of any good analysis. Get used to it.

[seycyrus]

As far as I can tell I answered all your past questions. Feel free to point me to one you are still awaiting an answer for. And the "not for me" phrase should not come as such a big surprise. Both damned and myself are dissecting FG's random number generator results in order to find evidence of how good or bad it works.

Err ... ok. I see a trend where you are going to have him provide multiple datasets of 10s of millions of rolls.


I am not sure if damned is trying to prove how good it is working against opposing views, but it seems like that.

No, trust me. From personal experience damned is working towards true enlightenment by calmly dealing with folks like you and me.


Personally I am just interested in quantitative and qualitative interpretation of real results,...

Well, I tried to entertain a discussion on the qualitative interpretation of the results, but you didn't follow up with discussion. How do statistical flaws in random number generators practically manifest in a system where the application of those random numbers is itself random?


Being inquisitive and critical in all directions is the basis of any good analysis. Get used to it. *I* am used to it.

[Weissrolf]

you could roll a million times and i would not expect the average to be 3.5
likewise i would not be surprised if it did equal 3.5
It would get closer and closer of course. Let me quote

https://www.crabfragmentlabs.com/s/Probability-For-Game-Designers.pdf


For example, the expected value of a single d6 is the average of all equiprobable results (1, 2, 3, 4, 5, 6), or 3.5. This is clearly not the most likely result (There is no 3.5 on a d6), just the average of all likely results. If you roll a d6 a million times and average the results, the value will be very close to 3.5. If you make ten million rolls, the value will be even closer.

[damned]

I am not really trying to prove anything.
I additionally dont even care if the dice are not perfect.
From all my observations the dice are easily good enough but that is very much my perspective.
I worry about the chance that a glitch occurs in game and spoils rolls - as some people have felt this has occurred to them - but Ive never seen it myself.

I dont think in real life dice roll fair ever.

[damned]

It would get closer and closer of course. Let me quote

https://www.crabfragmentlabs.com/s/Probability-For-Game-Designers.pdf

That is theoretical modelling.
If you roll 6 6s in a row the next d6 doesnt care and will still give you an equal chance of adding another 6 to the run.
The theory is good. And its likely to hold out.
But if it doesnt hold out on any particular run of dice rolls it neither invalidates the theory or the rolls.

[Weissrolf]

Err ... ok. I see a trend where you are going to have him provide multiple datasets of 10s of millions of rolls.
You are being snarky. As long as others don't have access to the rolling extension all we can do is make use of the numbers damned provides. The thread's whole idea is about providing logged numbers in a qualitative way.


No, trust me. From personal experience damned is working towards true enlightenment by calmly dealing with folks like you and me.
A trait we share then. I like that.


Well, I tried to entertain a discussion on the qualitative interpretation of the results, but you didn't follow up with discussion. How do statistical flaws in random number generators practically manifest in a system where the application of those random numbers is itself random?
What do you mean by "the application of those random numbers is itself random?"

I can tell you that some of my players insist that algorithmically generated pseudo-random numbers are not independent of the numbers before them. From what I have read some forum members also seem to suspect this. So any roll determines the outcome of successive rolls and thus rigs the system. One forum member here used the term "locked high" for some dice rolls.

These can be practically relevant problems for people basing a whole evening's tactical gaming results on said pseudo-random rolls. Even more so when they are used to play in socially more pleasing at the table situation, but are now forced to strip back the experience to the equivalent of an online computer game.

Comparing pseudo-random RNG (non random seed) to random.org's supposed true random RNG (random seed) should give some insight into the differences and non-differences of the results.

[Weissrolf]

That is theoretical modelling.
If you roll 6 6s in a row the next d6 doesnt care and will still give you an equal chance of adding another 6 to the run.
This is obvious, but with so many rolls you either get a closer approximation the theoretical model or check again (and again) if nothing is wrong with your RNG.

[damned]

You will either have to write one or wait till I push a new build of MoreCore and then everyone can have it.
Im sure that someone could write some much better code than me and get it to do more math but I too have work and family that calls.

Its currently in the process of throwing a million d6s
I wonder if FGU and my computer will handle it...

[damned]

This is obvious, but with so many rolls you either get a closer approximation the theoretical model or check again (and again) if nothing is wrong with your RNG.

Each time you state that it will go towards the mean I will reply back and say that it doesnt have to.
And that is statistically supportable even on a perfect generator.

[Weissrolf]

I worry about the chance that a glitch occurs in game and spoils rolls - as some people have felt this has occurred to them - but Ive never seen it myself.
In Pathfinder 2 this glitch would affect those characters more that rely more on rolls than others (the Fighter class would be the least affected). And that is in a system that by design relies more on rolls than its predecessor (much lower boni, more dice based boni, rather high chance of failing a role).

One of my players is playing the PF2 Ranger class in one campaign and the Monk class in another. They both use flurry of blows and thus work by using many attacks/rolls per round. It seems more disheartening to fails many rolls per round than just one roll per round, especially when you know that every result on the die has the same chance of appearing. After spending time on better understanding the new system I made clear to my players that this is how all classes but the Fighter work now and that every little bit (a single 1 hp more damage or a "measly" +1) can have a large impact on the number of rounds a fight lasts (like well over 20%).


I dont think in real life dice roll fair ever.
Here we are trying to make sure that at least the RNG/roll is equally unfair to everyone. And we did not even mention that on different PC hardware the RNG seed might differ and thus benefit or handicap single players. I really would like to know what seed (hardware event) is being used, especially knowing how random.org uses atmospheric noise (quite fascinating).

[Weissrolf]

You will either have to write one or wait till I push a new build of MoreCore and then everyone can have it.
No hurry here. After having to manually enter those first 600 rolls into Excel everything else is an improvement. "You have to write one" is something I would like to read less often on these forums. Not everyone aspires to be a coder, especially not for these kind of minor problems that only have relevance for less than 4h a week. We non-coders have other qualities that can improve peoples' lives (among other things I test drivers, hardware and software for developers and users for example).


Its currently in the process of throwing a million d6s
I wonder if FGU and my computer will handle it...
Ouch. I don't know if these can sensibly be imported to Excel, though. I still have to use mouse-dragging to spread the formulas over thousands of lines, no idea how this could be automated. The free version of random.org only allows 10k rolls per run, bringing those together to 100k was manual work already. Your code will calculate the average anyway, so that's covered already.

[seycyrus]

You are being snarky.

Snarky? I was simply stating the trend. Has the size of the datasets that Damned has provided been increasing or decreasing? Have you shown any inclination to state A) "ahh, that sample size is large enough", or has it been more along the lines of B) "more rolls needed!" ?


...The thread's whole idea is about ...

No, not quite. That might be where you have started to steer it, but the thread was originally about "Hey there, I'm SW guy. Give me your chat logs and I'll tell you how many times Drak the destroyer dropped his sword! Would you be interested in paying money for such a service?"


A trait we share then. I like that.
Damned is much farther along the path.



What do you mean by "the application of those random numbers is itself random?"

Aha! A discussion of sorts. Who determines how those numbers are used? For the player, or against the player? In a fight, or selling (or buying) a rose? Generating a random treasure (where just maybe - that natural 20 is the worst result!)

Real people interacting in a real game where very little (if anything) is predetermined determines how those random numbers are being used. The application of the "flawed" random number generators is randomly applied!


I can tell you that some of my players insist that algorithmically generated pseudo-random numbers are not independent of the numbers before them. From what I have read some forum members also seem to suspect this. So any roll determines the outcome of successive rolls and thus rigs the system. One forum member here used the term "locked high" for some dice rolls.

I can tell you that such commentary disregards the reality that real people are rolling the dice for totally unpredictable reasons.

Specifically as for "locked high", I have not seen anything in your posts that would indicate that you are looking at abnormal end distributions.


These can be practically relevant problems for people basing a whole evening's tactical gaming results on said pseudo-random rolls. Even more so when they are used to play in socially more pleasing at the table situation, but are now forced to strip back the experience to the equivalent of an online computer game.

No, it's exactly the opposite. Assuming of course that you and your players are people, and not AIs.


Comparing pseudo-random RNG (non random seed) to random.org's supposed true random RNG (random seed) should give some insight into the differences and non-differences of the results.

Since the application of the random number generators is random, the difference is irrelevant.

[damned]

Randomness can (and maybe should) be measured in many different ways.
You have suggested three good ones already - average of the rolls, doubles, and longer repeating runs.
What about where the dice land on the screen/table?
What about which direction the face faces?
What about the longest runs between same numbers?
There is a huge number of other number patterns that could be tested for.
What if the result came out with a perfect 3.5 average but the double numbers were badly skewed?
How would that effect your perception of the rolls legitimacy? <- serious question

Im not a coder. Im terrible at coding and I only get where I get by persistence (and leveraging off the brains around me at times).


"You have to write" one is something I would like to read less often on these forums.
if someone doesnt write it how will it happen?
the present already belongs to programmers - I only see this increasing in the future
i think that most people would personally benefit from learning some coding

[seycyrus]

...The free version of random.org only allows 10k rolls per run, bringing those together to 100k was manual work already. Your code will calculate the average anyway, so that's covered already.

Woah! *DING DING DING DING* "Calling Dr. PhD in statistics and probability, calling Dr. PhD in statistics and probability"

Do *we even know* that combing 10 10k runs should give the same sort of distribution as 1 100k run?

[Weissrolf]

No, not quite. That might be where you have started to steer it, but the thread was originally about "Hey there, I'm SW guy. Give me your chat logs and I'll tell you how many times Drak the destroyer dropped his sword! Would you be interested in paying money for such a service?"


...one of the recent discussions about dice rolling high or low for players prompted me to build...


Damned is much farther along the path.
I did not realize that you know us on such a personal level. Let's hug.


Who determines how those numbers are used? For the player, or against the player?... ... The application of the "flawed" random number generators is randomly applied!
Not if the "flawed" generator creates consecutive numbers that are biased in one direction or the other. Then the person who does several throws in a row is non randomly affected. The same applies when the flawed generator relies on flawed hardware seeds only on some players' computers where the hardware source differs from other players' hardware.

All that being said, proving FG's RNG *not* being flawed is just as useful as proving it flawed. The latter leads to software improvements (or bug fixing), the former leads to ease of mind for the players and a more definite stop to the discussions popping up again and again. Some people always feel threatened or attacked by others analyzing and potentially falsifying seemingly established systems. But getting "hey, this truly is working as intended" results are good, especially if questions keep coming up.


Specifically as for "locked high", I have not seen anything in your posts that would indicate that you are looking at abnormal end distributions.
I thought that looking at doubles was a start. And I also started playing with graphs to make distributions more easily visible. Not easy to do with so many numbers and the limits of Excel.


Since the application of the random number generators is random, the difference is irrelevant.
So why even bother with the quality of RNG, yes?! Ok, fair point. Statistics are useless then.

[Weissrolf]

Woah! *DING DING DING DING* "Calling Dr. PhD in statistics and probability, calling Dr. PhD in statistics and probability"
Oh, please.


Do *we even know* that combing 10 10k runs should give the same sort of distribution as 1 100k run?
When rolls are truly random then we know that every result in the 10x10k runs has the same chance of appearing as every result in the 1x100k run. When they are not truly random then indeed combining multiple runs could be different to a single large run. With a bad seed source you could even end up with two runs being the same (not likely to happen nowadays).

This is why random.org uses atmospheric noise as seed. And this also is why governments around the world start(ed) offering free true random number generators to the public: to offer a reliable and trustworthy RNG to either work with directly or to test your own RNG system against.

[damned]

Woah! *DING DING DING DING* "Calling Dr. PhD in statistics and probability, calling Dr. PhD in statistics and probability"

Do *we even know* that combing 10 10k runs should give the same sort of distribution as 1 100k run?

Well Im not dropping 1million dice in one handful either right?

[seycyrus]

Is there a reason why you continue to ignore the fact that whether or not it is the DM, or player A, or player B, or player C ... who is rolling the dice for reason 1, or reason 2, or reason 3 ... is itself random?

Not to mention that the dice mechanics behind the interpretation of the results is not predetermined? I play Gurps. You want to roll low, at least to hit.

[Weissrolf]

A botched system does not roll "too high" or "too low" all the time, it does so in sequences. And most dice rolls are done in tactical combat where dozens of rolls are done to determine the same outcome ("do I hit?"). Characters/Game systems with large fixed boni to rolls are less affected from RNG flaws than those with low fixed boni.

Anyway, if you insist that this all does not even matter then I don't see why we should have a thread about "Game Statistics". My suggestion to ddavison and damned would then be: spare yourself the work! Same suggestion to myself.

[seycyrus]

...

When rolls are truly random then we know that every result in the 10x10k runs has the same chance of appearing as every result in the 1x100k run. When they are not truly random then indeed combining multiple runs could be different to a single large run.
...

Are the distributions the same? Are the distributions of 100 1k runs with different seeds the same as 1 100k run? I don't know. Do you? Do you REALLY?

That's why I called for a Dr. in statistics and probability. Something that you took as yet another reason to either pretend to be offended or be offended.

[damned]

ok thread is closing for 24 hours :)
go roll some dice!

[ddavison]

I will admit that I have zero interest in proving or disproving someone’s idea of what a perfect distribution would be because as has been stated, there are too many other random elements injected into the mix to make that whole exercise pointless. After a life long passion for gaming, I mostly see poor tactical gamers blame dice rolls for poor results. I play Bloodbowl and Warhammer, that is notorious for players blaming the dice instead of their poor risk assessment skills.

Knowing how many goblins were defeated, total damage dealt, number of times someone was attacked and even how many times each person had to roll the dice before they got a crit are interesting to me. I suppose the better name for this thread would have been Campaign Game Analytics instead of Statistics.

Just because the discussion of dice roll statistics and RNGs doesn’t appeal to me, others may be more interested in that discussion. If 10k rolls has a mean of 3.45 every time, then that might be noteworthy. If it is 3.45 one time and 3.55 another time, then suddenly you are back to where you perceive it needing to be. Most computer RNGs factor the clock into calculations for seeding, so rolling 100 dice in a row is unpredictable and dependent upon other items.

When players are involved, they are often complaining about the results of less than 100 rolls in any given night spread out over several hours. These same players would complain about random.org generated numbers as well. You can’t convince them that the universe isn’t conspiring against their well laid plans or that their overwhelming success another night isn’t merely the rightful result of their brilliant and daring plan.

[damned]

At first glance on 1million rolls 2 might be overweight and 5 underweight
But you probably want to roll a few million more to be sure :)



#1 Rolled: 165801 Occurrence: 16%
#2 Rolled: 177194 Occurrence: 17%
#3 Rolled: 164340 Occurrence: 16%
#4 Rolled: 173570 Occurrence: 17%
#5 Rolled: 155657 Occurrence: 15%
#6 Rolled: 163438 Occurrence: 16%
Total Number of Rolls: 1000000
Sum of all Rolls: 3466402
Average of all Rolls: 3.466402
Actual Rolls in Order:
3, 2, 6, 4, 2, 1, 5, 5, 5, 1, 3, 5, 5, 6, 3, 6, 4, 4, 6, 1, 4, 5, 4, 2, 1, 3, 1, 2, 4, 4, 1, 4, 2, 3, 3, 3, 2, 2, 4, 2, 1,

Im out of rolls for a while.
When the next build of MoreCore comes out you can have the diceroller :)

[Weissrolf]

I will admit that I have zero interest in proving or disproving someone’s idea of what a perfect distribution ...
Knowing how many goblins were defeated, total damage dealt, number of times someone was attacked and even how many times each person had to roll the dice before they got a crit are interesting to me....
For me it may be the other way around. Once I know that rolls distribution can be trusted I generally don't care for after-game statistics, at least for normal gaming sessions.


After a life long passion for gaming, I mostly see poor tactical gamers blame dice rolls for poor results. I play Bloodbowl and Warhammer, that is notorious for players blaming the dice instead of their poor risk assessment skills.

Last months I did a whole bunch of rolls for my PF2 player chars versus a NPC to better understand the system and its implications. This enabled me to better advice my players that Pathfinder 2 favors dice-rolls over large fixed boni and the specific classes' position in the great picture of things. Small fixed boni are much stronger now, especially if you manage to stack them, but all within much more sane boundaries than PF1. The game-designers should have emphasized this philosophy in the core rulebook instead of some Gencon video I was just told about after doing the whole ordeal.


Just because the discussion of dice roll statistics and RNGs doesn’t appeal to me, others may be more interested in that discussion. If 10k rolls has a mean of 3.45 every time, then that might be noteworthy. If it is 3.45 one time and 3.55 another time, then suddenly you are back to where you perceive it needing to be.
Indeed, and this is why I asked for more rolls, to see how results develop over time (2x 3.45 looked fishy, so we needed more for comparison).

I still am a bit suspicious at how far off the theoretical mean damned's 1 mio result. 0.0336 seems quite much after so many rolls. Random.org manages such a high deviation only in *some* of its 10k rolls, which seems to quickly go down to less than 0.002 after 100k rolls. How much this matters in practice is not answered by this single criterion, it just tells us that there is a quantitative difference between FG and a high quality RNG.



Most computer RNGs factor the clock into calculations for seeding, so rolling 100 dice in a row is unpredictable and dependent upon other items.

https://docs.microsoft.com/en-us/dotnet/api/system.random?redirectedfrom=MSDN&view=netcore-3.1
"On most Windows systems, Random objects created within 15 milliseconds of one another are likely to have identical seed values."

Not a problem in practice, because we usually don't roll dice within 15 milliseconds in FG. Maybe damned MoreCore script does, though, I don't know.


When players are involved, they are often complaining about the results of less than 100 rolls in any given night spread out over several hours. These same players would complain about random.org generated numbers as well. You can’t convince them that the universe isn’t conspiring against their well laid plans or that their overwhelming success another night isn’t merely the rightful result of their brilliant and daring plan.
I now told my PF2 players to "expect to regularly *not* hit, especially not critical". Hopefully they will suffer less now, especially if 4 rolls in a row miss the target. That being said, one player still shouted in anger (at his rolls/bad luck) last time we played and yes there were many many many D20 results below 10 this time around.

Systems like D&D/PF suffer from the fundamental design philosophy that characters are expected to regularly not hit their target difficulty (be it combat or not). In practice (at the table) this means that some players miss so often that they feel useless or at least not as if their presence made a difference. Many small hits can feel more gratifying for some players, while others like the seldom big hit. Unfortunately all the game-breaking optimizations of the last years made players expect to often hit big, instead of often small vs. seldom big.

[Weissrolf]

Im out of rolls for a while.
When the next build of MoreCore comes out you can have the diceroller :)
Thanks damned, I may (likely) look into it once it's out. ;)

[Weissrolf]

Side-note: the roll6-1m.txt file only included 400k rolls, not 1 mio. My guess is that the log/file was inadvertently truncated.

[Weissrolf]

Unity increases the Windows timer resolution from 15 ms to 1 ms. While this might seem good (if a new seed is taken every roll) it is excessive for other purposed (laptop battery time).

If some moderator is reading here: I suggest to split the roll/RNG statistics discussion into its own thread, because we kind of hijacked ddavison's original campaign analytics intend.

[LordEntrails]

If some moderator is reading here: I suggest to split the roll/RNG statistics discussion into its own thread, because we kind of hijacked ddavison's original campaign analytics intend.
Done, enjoy :)

[Lonewolf]

Im not a coder. Im terrible at coding and I only get where I get by persistence (and leveraging off the brains around me at times).

Just a quick reminder in case you are not aware. There was an extension written for Savage Worlds that measures the dice results. So there is some code already done for FG that you can look at. It was designed to give Savage players a bit of a laugh rather that for statistical research. However if you interested in coding something more it might worth your time to explore.

[damned]

Side-note: the roll6-1m.txt file only included 400k rolls, not 1 mio. My guess is that the log/file was inadvertently truncated.

my guess is that the string value i was storing the rolls in hit a limit and no new data was being retained.
removing the commas would increase the amount of actual data retained but still not enough for 1million results
probably best to stick to 100k rolls im guessing
it took (guessing) 14 hours or more to do the million rolls

[Zygmunt Molotch]

Unity increases the Windows timer resolution from 15 ms to 1 ms. While this might seem good (if a new seed is taken every roll) it is excessive for other purposed (laptop battery time).

this is interesting

if there's going to be errors or issues (at all) I'd wager it'd be in the clock sampling/seeding, and they do use a seed, you're correct, some just don't understand the nomenclature of what they're describing :)

I'm not so sure the error is likely to be in the 'physics' modelling, but certainly whether or not the seed is refreshed/dropped/variable is pointing at the right thing...

with due respect to smiteworks, it's happened before with FGU, only last year, in 2019, so it's not entirely impossible that could happen... (d4s rolling zero...)

and if there is any issue, looking at a distribution like damned is providing is only going to reveal so much, and still could obfuscate actual problems (until it's drilled down as to what the seed was for each roll, or where the seed was sampled/was the variable reset etc)

personal dismissable anecdote: One person in a game I was in last week rolled 7 ones in a row. The odds of that are phenomenally low (and yes, law of large numbers, that can occur... but that's a cheap get out). The whole session was beset by numerous ones and twenties, more than any other result combined. Eyebrows were raised more than once.

not that its needed, but one of my masters projects was utilizing RNG modelling, and then I went to do atmospheric physics modelling, then finally financial modelling

[Weissrolf]

If a new seed is taken from a timer with every roll then increasing the Windows timer resolution is a good thing for random rolls (=decrease the time between timer updates). At the default of 15.625 ms it could happen that you do two rolls within the same timer period (=same seed). Not a big problem if the RNG algorithm keeps creating new results out of the same seed, but a problem if the same seed always leads to the same result.

That being said, damned average for 1 mio rolls was 3.467. This means that on average there were more rolls between 1-3 than there were rolls between 4-6. Random.org manages to come closer to the "ideal" average of 3.5 with less rolls being made, which suggests better distribution of results.

Once I get my hand on MoreCore I will do some more rolls of my own. It's also a stress-test for FGU stability. ;)

[damned]

personal dismissable anecdote: One person in a game I was in last week rolled 7 ones in a row. The odds of that are phenomenally low (and yes, law of large numbers, that can occur... but that's a cheap get out). The whole session was beset by numerous ones and twenties, more than any other result combined. Eyebrows were raised more than once.

So... why would any RNG seed weight itself to both 1s and 20s? How would the seed even know that there were 20 possible outcomes?
Can you share your chat log?

[Zygmunt Molotch]

Can you share your chat log?

I was just a player, sadly


So... why would any RNG seed weight itself to both 1s and 20s? How would the seed even know that there were 20 possible outcomes?
Can you share your chat log?

who can say, we're talking about a black-box without any information on how it works, asking me to explain a black box I didn't build is slightly disingenuous :D (ie: that's not my responsibility)

hypothetically because if the seed isn't changing, and the 'physics' model is what it is, a function, the result will be exactly the same...

but without extra info, it's not even safe to hazard a guess

[Zygmunt Molotch]

Not a big problem if the RNG algorithm keeps creating new results out of the same seed, but a problem if the same seed always leads to the same result.



if

it's been said it's a physics model, and thus likely a simple function, likely with one initial seed, but we're just guessing without details, from there on...


you're totally right, there's no verifiability (or possible improvement) without transparency, however

they don't have to provide any info, of course, that's SW's prerogative

from an engagement and outreach perspective it might be sensible to do so however, to quell or quash any rumours or distrust, but that's their decisions :D we can but beg!

[damned]

There is 400,000 consecutive d6 rolls in this post if you want to go pattern hunting.
https://www.fantasygrounds.com/forums/showthread.php?63411-Randomness-and-Analytics-of-Die-Rolls&p=554378&viewfull=1#post554378

[Weissrolf]

I think we are mixing things up here.

Seed: A truly random seed can guarantee a truly random RNG result, depending on the algorithm used. This is why atmospheric noise is used as seed for random.org. Personally I don't know if atmospheric noise is truly random, but it likely comes closer than mouse-movement (only used for fast enough movement) or the linear system timer.

RNG algorithm: The algorithm used for creating random numbers can either create a) a long sequence of numbers (2147483647 in C#) out of a single seed or b) a single random number out of every seed.

a) The quality of the list of numbers depends on the algorithm being used and you can find tons of internet discussions about the various pros & cons of different approaches (one being performance).


Pseudo-random numbers are chosen with equal probability from a finite set of numbers. The chosen numbers are not completely random because a mathematical algorithm is used to select them, but they are sufficiently random for practical purposes. The current implementation of the Random class is based on a modified version of Donald E. Knuth's subtractive random number generator algorithm.


I'm not sure what algorithm is used to generate Unity's random numbers, but C#'s Random class is, according to the documentation, using an algorithm based on a modified version of Donald E. Knuth's subtractive random number generator algorithm. So there's more than one type of algorithm generating pseudo random numbers. If you study the area you will learn that the Mersenne Twister is considered the best pseudo random number generator.


The Mersenne Twister is a pseudorandom number generator (PRNG). It is by far the most widely used general-purpose PRNG.[1] Its name derives from the fact that its period length is chosen to be a Mersenne prime.

b) Creating several RNGs at once can be problematic, because they may all have the same seed when the Windows system timer resolution is too coarse (FGU uses 1 ms, which is close to the minimum of 0.5).


The default seed value in the .NET Framework implementation is equal to the value of the Environment.TickCount property. Its resolution is limited to the resolution of the system timer (typically in the range of 10 to 16 ms). Thus, using Environment.TickCount has an unfortunate consequence if you create multiple instances of the Random class at once

Just taking the first number out of each seed's sequence (and forgoing the rest of the sequence) can be problematic, too, because the correlation between seeds is said not to be random.


Can’t I just use random number generators with different seed values?

No! This is a very common misconception about RNGs. The fact is that while the different numbers in the same sequence are random in relation to each other, the same indexed numbers from different sequences are not random in relation to each other, even if it may look like it at first glance.
...
So if you have 100 sequences and take the first number from each, those numbers will not be random in relation to each other. And it won’t be any better if you take the 10th, 100th, 1000th number from each sequence.
...
Though the pattern is rather uniformly distributed, it isn’t quite random. In fact, I’ve shown the output of a purely linear function for comparison, and it’s apparent that using numbers from subsequent seeds is barely any better than just using a linear function.

And this is the point where it gets interesting for our purpose. Frankly, we don't care so much how "truly" vs. "practically" random our rolls are. What we care about is how random their distribution is within a practical time frame: preferably 1 single game-session or at least as few sessions as probable!

Within this single game-session we do not want 1) subsequent rolls to be in relation to each other and we do want 2) a somewhat reasonably (as in statistically probable) uniform distribution of rolls without (improbable) patterns.

1) means that the first roll should have no effect on the second roll, should have no effect on the third roll, should have no effect on the fourth roll...

2) means that we expect to see all numbers between 1-20 on a D20 being somewhat well represented within a bunch of session or hopefully even a single session. Yes, long sequences of equal results are possible in a truly random sequence, but that's not our main concern. We are more concerned about dice results repeatedly leaning on the low, high or middle spectrum of the range, as in patterns (or waves).

Extreme examples:

1+1+1+20+20+20 = ave. 10.5
1+2+3+18+19+20 = ave. 10.5
10+10+10+11+11+11 = ave. 10.5
8+9+10+11+12+13 = ave. 10.5

All of these examples satisfy the statistical average, but obviously their distribution is completely bonkers. The former two favor extreme lows and highs, the latter two favor the middle. Yes, it can happen and is even expected to happen sometimes, but over a sensible amount of time (game-sessions) it should not go only back and forth between extremes, aka there should be no determined patterns!

[damned]

So after looking at the 400,000 rolls what is your opinion?

[Weissrolf]

My opinion is that the average of your 1 mio rolls should be closer to the statistical average to not favor one side of the dice results (lows vs. highs). Other than that I lack the scientific/mathematical background to analyze the number for randomness and patterns. I checked various software solutions, but they need to be programmed, except for one that is available as pre-compiled binary.

https://www.fourmilab.ch/random/

For the latter I am still trying to figure out how to feed it the number results of your rolls, because feeding it the text file results in the binary file data being analyzed, not the text number data content.

All that being said: SW could just publish how they create random numbers (I suspect by simply initiating .NET's random class), so those who care know what they are working/gaming with.

[Zygmunt Molotch]

There is 400,000 consecutive d6 rolls in this post if you want to go pattern hunting.
https://www.fantasygrounds.com/forums/showthread.php?63411-Randomness-and-Analytics-of-Die-Rolls&p=554378&viewfull=1#post554378

as explained, pattern hunting is irrelevant without knowing how these were generated

400,000 using the same seed
400,000 using the default seed (there has to be a default when the "go get seed" function breaks)
400,000 using zero
400,000 using sequential numbers in 1 mil second increments
400,000 using sequential numbers in 1 second increments

all may give different distributions

certainly it will tell you how good or not your session was :)

but if something is broken _elsewhere_ it'd be worthless

there's so many potential horrors of how it could be wrong :D thats the scary thing about black boxes

[Kelrugem]

personal dismissable anecdote: One person in a game I was in last week rolled 7 ones in a row. The odds of that are phenomenally low (and yes, law of large numbers, that can occur... but that's a cheap get out). The whole session was beset by numerous ones and twenties, more than any other result combined. Eyebrows were raised more than once.

One anecdote from me at a real table with a real d20: I once rolled eight 1s in a row... :D (EDIT: Aaah, I meant a d6, not d20! :D Accidentally wrote d20 due to your anecdote :D (though seven 1's of a d20 in a row is extremely cool as result, your player should play lottery :D So, assuming that everything is indeed going well in FG, your player should mark that day in their calendar, won't happen that quickly again :D))

I understand that one wants to study the background of the code/seeds whatever, but taking such anecdotes as a proof of that something is going wrong is mathematically not correct (though you got indeed a very extreme result :D). But I am not saying that you said that this is a mathematical proof, you said the opposite; I am just saying it because some people see something like this as a proof of something going wrong, while living on a planet in the far universe with life on it :D Our brains are simply rarely able to think statistically or exponentially or whatever, maybe only linear stuff etc. :) Hence, why there are people like me study e.g. such things as their profession (mathematician) :D

I cannot completely remember, but was is not YouTube and/or Apple who have coded their shuffle programmes in such a way that "extreme" results do not appear, such that their shuffle programme was not really that random, and the reason for this code was because they wanted to stop that people post extreme results of the shuffle for claiming that the shuffle is not "random enough", although such things can happen (especially when a lot of people use that)? :D (not sure whether I remember that story correctly, so, take that with a grain of salt)

Nevertheless, surely interesting to study :) But difficult without further information about what FG does :)

(I also doubt that their physical die is a "simple function", would be new to me that someone explicitly solved the differential equation for the equation of movement of dice; I rather expect that SmiteWorks may use some provided simulator which (pre-)calculated the movement of dice numerically :) At least that is how I always understood the "physical" in FG's ad :D But who knows, there are real masters of solving differential equations out there :D But I am also no expert in how gaming engines do that kind of stuff, I may make the mistake to extrapolate from my profession :D )

[LordEntrails]

And this is the point where it gets interesting for our purpose. Frankly, we don't care so much how "truly" vs. "practically" random our rolls are. What we care about is how random their distribution is within a practical time frame: preferably 1 single game-session or at least as few sessions as probable!

Within this single game-session we do not want 1) subsequent rolls to be in relation to each other and we do want 2) a somewhat reasonably (as in statistically probable) uniform distribution of rolls without (improbable) patterns.
But that doesn't sound random :)

But I also get it. So what do "we" want? No more than 2 or 3 sequential rolls? What about sequential rolls by different users? It is ok to have 3 "6"s rolled sequentially if they are from different users, methods, actions but not if they are from the same method?

I do find the discussion interesting, but now we are saying that we don't really want the dice to be random, because then their will be edge cases that 3 or 4 or ten times in a row a given value happens sequentially, but then again forcing such not to happen means we don't want 'random' results.

And, does it matter? If an 'anomaly' (i.e. something that makes us uncomfortable) happens once per 100 gaming sessions is that ok?

And what role does confirmation bias play into what makes us uncomfortable?

As well, do we care to compare FG's method (whether the physics model or the random number generator/algorithm) to actual die results? (Which we know are NOT random.)



I cannot completely remember, but was is not YouTube and/or Apple who have coded their shuffle programmes in such a way that "extreme" results do not appear, such that their shuffle programme was not really that random, and the reason for this code was because they wanted to stop that people post extreme results of the shuffle for claiming that the shuffle is not "random enough", although such things can happen (especially when a lot of people use that)? :D (not sure whether I remember that story correctly, so, take that with a grain of salt
Most (all?) media players have alogrithms to determine "shuffle" play order, because as you note, people do not want a song/video to play twice in a row (or close to sequentially) because that might be random, but it's not an enjoyable "shuffle" experience :)

[Kelrugem]

Most (all?) media players have alogrithms to determine "shuffle" play order, because as you note, people do not want a song/video to play twice in a row (or close to sequentially) because that might be random, but it's not an enjoyable "shuffle" experience :)

ah oki, yeah, user experience sounds like a good argument in that regard :D though one of the other easy statistical models is where you can draw certain things just once (so, they do not happen twice etc.), was more thinking about succeeding series of music (like music 1, then 2, 3, 4..) :) But not allowing certain results can still be a statistical system of course, just not one of the "native/easy/canonical" ones we usually think about :) (especially not a "typical die") I just found the reason behind that code funny, although I may remember that one article wrongly of course :D

[damned]

as explained, pattern hunting is irrelevant without knowing how these were generated

400,000 using the same seed
400,000 using the default seed (there has to be a default when the "go get seed" function breaks)
400,000 using zero
400,000 using sequential numbers in 1 mil second increments
400,000 using sequential numbers in 1 second increments

all may give different distributions

certainly it will tell you how good or not your session was :)

but if something is broken _elsewhere_ it'd be worthless

there's so many potential horrors of how it could be wrong :D thats the scary thing about black boxes

I dont understand your point at all
Its irrelevant how the numbers were generated if you cannot find an issue with their randomness
Pontificating about theory is wonderful but if the actual results are random as far as the experience goes then its irrelevant
Academia is full of theories that have no bearing on real life

additionally you cannot roll 400,000 times in any short period of time so only the scenario of a non changing seed has any potential relevance

why are we talking about horrors and black boxes?
we are talking about dice rolls that may or may not impact the lives of figments of our imaginations
we lament the final bad roll that undoes us but ignore everything that leads up to it

here are the results from my game on Saturday night in order
different dice sizes rolled by different people
note that 7 times PCs were unconscious and 1 died and not one person questioned a single dice roll

[d20 = 16]
[d20 = 7]
[d20 = 14]
[d20 = 18]
[d20+2 = 16]
[d20 = 12]
[d20+7 = 23]
[g20+3 = 7]
[d20+4 = 8]
[d20+4 = 23]
[d20+2 = 18]
[d20+3 = 15]
[d20+2 = 22]
[d20-1 = 0]
[d20-1 = 11]
[d20-1 = 4]
[g20+5 = 20]
[d20+5 = 18]
[d20+1 = 9]
[g20+2 = 20]
[d20+5 = 16]
[d20+1 = 9]
[d20+4 = 15]
[d4+2 = 5]
[d20+4 = 19]
[d4+2 = 6]
[d20+5 = 15]
[d20+2 = 20]
[d20+3 = 19]
[d20+3 = 23]
[d20+3 = 22]
[d20 = 13]
[d20+4 = 8]
[d20+3 = 13]
[d20+4 = 22]
[d20+3 = 22]
[d20+2 = 10]
[d20+2 = 15]
[d20-1 = 0]
[d20-1 = 14]
[d20+6 = 21]
[d8+4 = 11]
[d20+5 = 23]
[d6+3 = 9]
[d20+4 = 6]
[d20+3 = 13]
[d20+4 = 23]
[d6+2 = 5]
[d20+3 = 5]
[d20+4 = 24]
[d6+g6+2 = 5]
[d20+4 = 12]
[d20+4 = 17]
[d4+2 = 5]
[d20+4 = 7]
[d20+4 = 18]
[d6+2 = 5]
[d20+5 = 9]
[d10 = 3]
[d20+5 = 16]
[d12+3 = 9]
[d20+6 = 25]
[d8+4 = 10]
[d20+5 = 24]
[d6+3 = 9]
[d20+4 = 6]
[d20+4 = 16]
[d4+2 = 6]
[d20+4 = 8]
[d20+5 = 24]
[d20+5 = 13]
[d20+3 = 6]
[d20+5 = 16]
[d20+3 = 4]
[d20+4 = 17]
[d20+2 = 18]
[d20+3 = 18]
[d20+3 = 23]
[d20+3 = 15]
[d20 = 12]
[d20+3 = 19]
[d20+4 = 10]
[d20+4 = 15]
[d20+4 = 22]
[d6+2 = 4]
[d6+2 = 4]
[d20+3 = 14]
[d20+4 = 19]
[d6+2 = 6]
[d20-1 = 5]
[d20+4 = 14]
[d20+2 = 12]
[d20+2 = 16]
[d20-1 = 1]
[d20+3 = 14]
[d20+4 = 14]
[d20+5 = 22]
[d10+3 = 8]
[d6+2 = 6]
[d20+3 = 21]
[d20+4 = 15]
[d20+4 = 20]
[d6+2 = 5]
[d20+6 = 10]
[d20+6 = 24]
[d20+4 = 20]
[d4+2 = 5]
[d20+5 = 24]
[d12+3 = 9]
[d20+5 = 24]
[d20+5 = 6]
[d20 = 15]
[d20+3 = 14]
[d20+4 = 22]
[d6+2 = 4]
[d20+6 = 23]
[d8+4 = 8]
[d20+5 = 18]
[d6+3 = 5]
[d20 = 6]
[d20+4 = 5]
[d20+5 = 9]
[d20 = 15]
[d20+5 = 9]
[d20+2 = 21]
[d8+6 = 8]
[d8+6 = 8]
[d20+3 = 15]
[d20+5 = 9]
[d20-1 = 8]
[d20+5 = 9]
[d2 = 1]
[d20 = 18]
[d20+3 = 5]
[d20 = 17]
[d20+4 = 22]
[d20+5 = 8]
[d20+5 = 23]
[d20+3 = 19]
[d4 = 3]
[d20+3 = 21]
[d20+2 = 5]
[g20+4 = 21]
[d20+4 = 24]
[d20+5 = 16]
[d6+3 = 6]
[d20+4 = 22]
[d6+2 = 7]
[d20+4 = 7]
[d20+6 = 9]
[d20+4 = 12]
[d20+4 = 9]
[d20+4 = 6]
[d20+5 = 21]
[d12+3 = 10]
[d20+4 = 20]
[d20 = 17]
[d8+2 = 7]
[d20+6 = 19]
[d8+4 = 8]
[d20+3 = 13]
[d20+1 = 17]
[d20+4 = 11]
[d20 = 17]
[d20+7 = 12]
[d20+4 = 10]
[d20+4 = 19]
[d4+2 = 3]
[d20+4 = 22]
[d6+2 = 3]
[d20+6 = 22]
[d8+4 = 6]
[d20+5 = 20]
[d6+3 = 9]
[d20+4 = 15]
[d4+2 = 4]
[d20+4 = 21]
[d4+2 = 4]
[d20+4 = 15]
[d20+5 = 23]
[d20+4 = 20]
[d12+3 = 8]
[d6+2 = 6]
[d20+3 = 9]
[d20+4 = 11]
[d20 = 8]
[d20+5 = 6]
[d20+4 = 6]
[d20 = 12]
[d20+4 = 16]
[2d8+2 = 14]
[d20+5 = 8]
[d20+2 = 6]
[d20+2 = 9]
[d8 = 2]
[d20 = 2]
[d20+6 = 13]
[d20 = 10]
[d20+4 = 8]
[d20+5 = 23]
[d6+3 = 7]
[d20+4 = 22]
[d6+2 = 8]
[d20+3 = 13]
[d20 = 17]
[d20+6 = 7]
[d20 = 7]
[d20+4 = 20]
[d20+4 = 6]
[d20+5 = 22]
[d12+3 = 14]
[d20+4 = 9]
[d20+4 = 7]
[d20+6 = 14]
[d20+4 = 18]
[2d8+2 = 8]
[d20+5 = 24]
[d12+3 = 10]
[d20+4 = 24]
[d8+g8+2 = 17]
[d20+4 = 5]
[d20+1 = 14]
[d20+1 = 4]
[d20+1 = 13]

[Weissrolf]

But that doesn't sound random :)
That is because you seem to suggest that random means "anything is possible", but the most probable outcome of a random sequence is uniformly distributed noise. There is no Shakespeare to be found in there, there is just noise.

Of course we usually don't have enough rolls in a single session to achieve that, which is why I wrote "or at least as few sessions as probable". The goal is that the RNG algorithm should not increase the number of sessions needed to get uniformly random distribution, aka noise.


But I also get it. So what do "we" want? No more than 2 or 3 sequential rolls? What about sequential rolls by different users? It is ok to have 3 "6"s rolled sequentially if they are from different users, methods, actions but not if they are from the same method?
There is no need to mess with results if the RNG rolls uniformly distributed random results in probable time. (Very) long sequences of the same number are a statistical exception, not the norm. With a good RNG we do not need to roll millions of results to get uniform noise. 10 ms (4410 samples) of white-noise always result in a hissing kind of sound (maybe attack like), but never in a sinusoid, it just doesn't happen.

Now, if we want to be subject to a random dice roll to such a large degree is an entirely different question. D&D5E and PF2 put more emphasis on the dice compared to fixed boni, which means players are more subjected to chance/possibility than certainty. Other systems put more emphasis on fixed boni or at least use something like 2D10 to create a bell-curve distribution of dice results.

[LordEntrails]

But random noise often has sequences of three in a row(or 2 or 4 or escalating sequences of limited length). Yet somehow we say that must not be random. You said we didn't want to see sequential rolls, but sequential rolls (5% of the time on a d20) is part of what a truly random generator would give.

Though I think the conclusion everyone here can take is simple, within the error that matters during a session or campaign, FG is more random than it needs to be.

[Weissrolf]

No, I did not say that we do not want to see sequential rolls, we don't want to see concentrations of improbable sequences.

We do not want the left kind of noise, we want the center and right kind of noise.

https://blogs.unity3d.com/wp-content/uploads/2015/01/HashComparison2.png

They all contain blotches of more concentrated black and white (aka sequences), but the left example is not properly distributed and contains more improbable concentrations.

[seycyrus]

Again I fear that this is another example of a little knowledge being not only dangerous, but a waste of time. Hell, I'm not even afraid, I KNOW.


That is because you seem to suggest that random means "anything is possible",

That IS what it means.


but the most probable
Seems that some of us aren't really sure what "most probable" means.


outcome of a random sequence is uniformly distributed noise. There is no Shakespeare to be found in there, there is just noise.
Yeah, you seem to circularly defining what you mean by "noise".


Of course we usually don't have enough rolls in a single session to achieve that, which is why I wrote "or at least as few sessions as probable". The goal is that the RNG algorithm should not increase the number of sessions needed to get uniformly random distribution,

What? An artificially forced uniform distribution is ARTIFICIAL.


(Very) long sequences of the same number are a statistical exception, not the norm.
If you never have very long sequences, then your distribution is forced, artificial and unrealistic.


With a good RNG we do not need to roll millions of results to get uniform noise. 10 ms (4410 samples) of white-noise always result in a hissing kind of sound (maybe attack like), but never in a sinusoid, it just doesn't happen.
Yeah, I guess you've never heard of sinusoidal noise.


Now, if we want to be subject to a random dice roll to such a large degree is an entirely different question. D&D5E and PF2 put more emphasis on the dice compared to fixed boni, which means players are more subjected to chance/possibility than certainty. Other systems put more emphasis on fixed boni or at least use something like 2D10 to create a bell-curve distribution of dice results.

Again, totally ignoring the fact that the rolls are applied randomly in random situations.

[Weissrolf]

Thanks for the contribution, I appreciate it.

[damned]

I really dont understand.
You refer to "uniform and random" together.
It does sound to me that you want something less random and more uniform.
You want a 1 for every 20 and and a 7 for every 13 and you want all the numbers covered off before we go thru too many rounds.
To me that doesnt sound random.
You say you dont want to see concentrations of improbable sequences but fail to show any.

We remember low numbers because they tend to hurt and they DO HAPPEN more often.

[Weissrolf]

One could even say that the definition of randomness is, in a way, a paradox. On the one hand, we say that a truly random sequence cannot conceal any rule that would enable us to recreate the sequence, while on the other hand, requiring the absence of all patterns within a sequence leads to a very restricting definition which is almost impossible to apply in practice. For something to be random, it must meet very well defined conditions. Randomness is thus defined by the complete absence of form which is, on the other hand, a very strictly defined form in itself, only with a negative sign.

White noise is random (frequency) values uniformly distributed over time (with a mean of zero). We want dice results to be as white noisy as possible (with a mean based on the specific dice).


A random process is a sequence of random variables whose outcomes do not follow a deterministic pattern, but follow an evolution described by probability distributions.

In this view, randomness is a measure of uncertainty of an outcome, rather than its haphazardness, and applies to concepts of chance, probability, and information entropy.

[damned]

Im curious where you got your definition of randomness from.
Genuine question.
I havent found a definition that links random and uniform together.
Same for white noise.

https://deepai.org/machine-learning-glossary-and-terms/white-noise#:~:text=White%20Noise%20is%20a%20random,inde pendent%20and%20have%20identical%20probabilities.

Most white noise references that include the word uniform seem to not be talking about truly random numbers.

[Zygmunt Molotch]

I dont understand your point at all
That's fine!

Its irrelevant how the numbers were generated if you cannot find an issue with their randomness
no it's really not

Pontificating about theory is wonderful but if the actual results are random as far as the experience goes then its irrelevant
I'm not trying to pontificate, I'm explaining very simply that a blackbox is unverifiable


Academia is full of theories that have no bearing on real life
thats a non sequitur, squire


additionally you cannot roll 400,000 times in any short period of time so only the scenario of a non changing seed has any potential relevance
not true, you have yet to establish the seed has even changed or is pointing at what it's supposed to be


why are we talking about horrors and black boxes?

because that's what a closed system with no transparency is


we are talking about dice rolls that may or may not impact the lives of figments of our imaginations

indeed we are, we're talking about a piece of code which is obfuscated, in my industries, both financial and atmospheric (and statistical) physics, that's the equivalent of a glowing neon red flag right there


we lament the final bad roll that undoes us but ignore everything that leads up to it

A truth, indeed, that's just confirmation bias, and we're trying to remove bias' here, all of us :)

because; while we have different approaches, and perhaps lingual quirks, we all love RPGs and dice rolling :)

let's work together for unveiling, rather than debate

perhaps then, we get some clarity

put it this way, it's hardly a trade secret

[damned]

So what point are you trying to make?
You want to know what the details of the seed are.
The devs have declined to share that info.
In the meantime - I cant see any evidence that there is an issue here.

We are not talking about a piece of obfuscated code.
That is misleading.
Obfuscated code is code that would otherwise be readable but is obfuscated.
This is code within a compiled application.
Not the same thing.

[Zygmunt Molotch]

You want to know what the details of the seed are.
The devs have declined to share that info.


Then this issue never goes away



We are not talking about a piece of obfuscated code.
That is misleading.
Obfuscated code is code that would otherwise be readable but is obfuscated.
This is code within a compiled application.


very much the same thing in industry, it's the core functionality of the program and just asking clients to 'trust' is weak

There is nothing we haven't seen before, in terms of RNGs, there is no uber secret new generator.

My point, is without transparency or code review (perhaps a well put together show and tell, explaining how good it is), this will never go away.

it's happened before with FG and still occurs that variables are undeclared/uninitialised or point at the wrong thing (everybody wince, we've all done it, and everyone always does it :D we are after all only human) ...

I'm not saying the house is on fire, I'm saying we've smelled burning electronics more than once

[damned]

No ones telling you that the system is perfect.
Im just waiting for someone to share data that shows an issue.
Will you share your data?

Additionally - are you suggesting that when you worked in the financial industry you or your business had access to source code for all software you used?
Unless you or your business wrote the software that is highly improbable.
Especially in the financial industry.

[Zygmunt Molotch]

No ones telling you that the system is perfect.

but that's what you're doing, by erroneously defending the indefensible, I'm not sure why either, you love FG, so do I, it needs to be better, lets make it better, community ad astra, etc


Will you share your data?
what data? I want to see the RNG mechanism, that's all, to make it better, and quell/quash suspicion


Unless you or your business wrote the software that is highly improbable.
That's exactly what I do :D it is rather fun, and gives me enough time to prat around prepping GM games :D

[damned]

but that's what you're doing, by erroneously defending the indefensible
How? I present data and the data has yet to show any smoking gun.
When 1,000 rolls was deemed too few I provided 10,000. Then 100,000 then 1,000,000.



what data? I want to see the RNG mechanism, that's all
Its easier to disprove than prove validity.
Yet no one has done so.



That's exactly what I do :D
So look at your computer.
Every application, driver and system on there is open source or written by your team.
Improbable.

[Zygmunt Molotch]

When 1,000 rolls was deemed too few I provided 10,000. Then 100,000 then 1,000,000.
I mean this, with due respect, with no rigorous control, or knowledge of the method (not your method , I mean the underlying code), this is irrelevant


Its easier to disprove than prove validity.

ie: the principle "it is claimed it's random and fine, by SW and the defenders" <- this is Burden of proof, the person issuing that statement has to justify or substantiate that, and since the 'defenders' are arguing from ignorance (of the underlying code) they're in an untenable position

Hitchens's Razor

although we have advanced the conversation slightly further beyond this, and conceived of an idea; that the code is sus', burden of proof STILL lies with SW


Yet no one has done so.
yet, bold :D I wouldnt venture that gambit just yet


Every application, driver and system on there is open source or written by your team. Improbable.

Facetious, lol, not much off the shelf stuff on our racks :) with maybe the exception of the distro, but enough about my coffee intake and penguin time mechanisms :D

[damned]

with no rigorous control, or knowledge of the method, this is irrelevant


although we have advanced the conversation , and conceived of an idea; that the code is sus', burden of proof STILL lies with SW

ie: the principle "it is claimed it's random and fine, by SW and the defenders" <- this is Burden of proof, the person issuing that statement has to justify or substantiate that, and since the 'defenders' are arguing from ignorance (of the underlying code) they're in an untenable position

Hitchens's Razor


yet, bold ;-)



Facetious, lol, not much off the shelf stuff on our racks :) with maybe the exception of the distro

You write code.
Write some code to roll some dice and search for repeatable anomalies.
Its not even that someone has presented evidence showing issues with randomness.
There are hundreds of thousands of games being played.
Every single one of those records the dice thrown in the chat.
There are millions of chat logs out there.

Even with the seed you could not prove true randomness - you can only disprove it.

[Zygmunt Molotch]

You write code.
Write some code to roll some dice and search for repeatable anomalies.


sure, I can whatsapp you my paypal, you may not like my hourly rates :D


Its not even that someone has presented evidence showing issues with randomness.
no one actually has to, the burden of proof that it's not flawed, lies with those claiming it's not

There are hundreds of thousands of games being played.
irrelevant

Every single one of those records the dice thrown in the chat.
no, incorrect, the totals are, not the rolls, you would need the RNG seed (for when there's no mouse entry) not passed through any 'kinetic model' to even begin to study, dice rolls, and totals aren't what is needed (at this stage) , you analyse every level, before filters or functions... and after

There are millions of chat logs out there.
irrelevant, unaccessible information, even with that, we'd need change logs, and the code; which has rather been my point...


Even with the seed you could not prove true randomness - you can only disprove it.

sort of, but you're again removing the burden of proof from SW

it's claimed it's not flawed, it needs to be proved, otherwise it can be dismissed without evidence, ordering matters ;-)

now we're going around in circles; no advancement

I offer the olive branch and suggest we break, no need to fall out over opinions on approach to evidence :) let's roll some dice and see what happens!

after all, the code, specifically the dice rolling code, has been shown to be borked at least once already! it'd be pretty bold to say there's no further issues...

[Weissrolf]

Im curious where you got your definition of randomness from.
From the links I provided and my personal understanding. We also have to clarify some thing:

- With dice rolls we are looking at "limited" randomness in that the range of possible results is limited (D20 = 1-20). Our randomness explicitly cannot contain just about any number, but only integers in between fixed boundaries.

- Dice rolling is a one-dimensional problem, hence it is not equal to the two-dimensional white-noise problem (frequency + amplitude). There are just similarities that made me use white-noise as discussion example and to express that we want our dice-rolling to be "more white-noise like". Interestingly white-noise comes in two variations where frequency is uniformly distributed over time, but amplitude can either be uniformly distributed or gaussian (bell-curve).


I havent found a definition that links random and uniform together.

First hit: https://en.wikipedia.org/wiki/Continuous_uniform_distribution


It is the maximum entropy probability distribution for a !!!random!!! variable X under no constraint other than that it is contained in the distribution's support.[3]

"Uniform" in the headline, "random" in the text body. As I am no expert I might mix things up, but currently I don't get the impression that I do.


Same for white noise.

https://deepai.org/machine-learning-glossary-and-terms/white-noise#:~:text=White%20Noise%20is%20a%20random,inde pendent%20and%20have%20identical%20probabilities.
From your link:


White Noise is a random signal with equal intensities at every frequency
To my understanding this means uniform distribution. If intensities are uniformly distributed over all frequencies then all you get is hissing noise. There are no more patterns to be recognized, it's random and formless. One reason why I chose white-noise as example is that like dice rolls you usually talk about audible/visible boundaries and part of the atmospheric noise used by random.org is made up of white-noise. And it can more easily be demonstrated audibly or visually, including patterns when it is not truly random and uniformly distributed.


defined in statistics as a signal whose samples are a sequence of unrelated, random variables with no mean and limited variance.
Limited variance reads like the boundaries set by amplitude/frequency minimum/maximum. I do wonder about the "no mean" part, though, because the mean of amplitude is 0 (as seen in their graph and later statement) and the mean of frequency should be around the Nyquist frequency (half the maximum frequency). Since I am neither Mathematician nor Physicist I may get this wrong, though, would be nice if one could chime in here.


... it may be required that the samples are independent and have identical probabilities.
This is what we want in unmassaged dice rolling. All rolls should be independent from each other and all results should have equal probabilities.

Again, I do *not* agree that "equal probabilities" means that everything can happen. There is virtually no likelihood that you will find Beethoven's 5th in there, you rather get uniform distribution of results over time with a mean close to the theoretical mean.


Most white noise references that include the word uniform seem to not be talking about truly random numbers.
I wonder how you come to that conclusion after reading the article you linked us to?


White Noise, by definition, works by defining parameters in which data is ensured to be random, unrelated, and have zero mean.

[Weissrolf]

How? I present data and the data has yet to show any smoking gun.
Here I have to intervene, because you are making it too easy for yourself and too hard for us. Testing randomness is a scientific achievement and the methods being made available by the scientific community demand a level of expertise that most of us do not possess. So saying "you did not falsify it yet" does *not* mean that there is nothing to falsify. At this point it only means that we lack the skills to use the "official" falsification methods.

https://en.wikipedia.org/wiki/Randomness_tests



Diehard
TestU01
ent utility from Fourmilab
NIST Statistical Test Suite (Special Publication 800-22 Revision 1a)
I mentioned earlier that I did not succeed yet to get "ENT utility" to accept your rolls as input data, but I tried. It lacks documentation and I did not wrap my head around how to input Unicode text numbers. It was used in the Unity blog article and its writer still felt the need to come up with his own randomness test mechanics on top of it to make patterns more visible. I put all of this in my posts already, links, images and quotes already. So just saying "Try harder!" is a bit disingenuous at this point.


When 1,000 rolls was deemed too few I provided 10,000. Then 100,000 then 1,000,000.
And until you publish(ed) your MoreCore update we other participants of this discussion (specifically !not! a complain-fest) even lacked the means to obtain the data necessary for scrutiny ourselves.


Its easier to disprove than prove validity.
Disproving pseudo-randomness is no walk in the park, unless the pseudo-random number generator is truly awful.

[Weissrolf]

Another word of clarification: At this point I do not even suggest that the rolling mechanism in FG is particularly erroneous or bad. The information published about the process seem rather hand-waved to me, though, which makes it harder to objectively interpret the results. What I asked for was more information or a scrutiny of results in order to better assess its quality.

I assume everyone partaking in the discussion to be benignly interested in the topic and its possible conclusion, personally I prefer discussing the topic instead of discussing the participants.

All pseudo-random number generators are flawed, it's by design, they may still suffice for our practical purposes. But there are better and worse implementations and in a hobby/program that fundamentally builds its mechanics (not enjoyment) on random numbers we strive for a better design. Furthermore, we (in this thread) strive to test and retest the RNG implementation in order to reveal obvious and hidden faults.

Asking for software (implementations) not to be put under scrutiny (even by end-users) is asking for trouble! Even more so when a new product "developed from the ground up" has just been officially published and the author does not share specifics of the implementation. What is shared about the implementation is that mouse-movement *may* be used in creation of the seed, but only if its sufficiently fast enough, else either a *fixed* minimum speed is set or mouse-movement or it's just discarded.

My original question about what seed is being used when no (sufficient) mouse-movement is present. The reason why mouse-movement is used as potential seed is that it's a more random seed than an linear timer, the goal is to increase entropy. In reality mouse-movement is smoothed out by drivers, sampled at slow intervals, reinterpreted by the OS and software and often enough rather repetitious. We don't even know if said mouse-movement seed is taken every time a roll is made or only once at the beginning. Heck, we even learned through linked articles that re-seeding every roll can lead to rolls *not* being independent from each other.

Using external sources of entropy for seeds can be a good idea, at least when those sources are really random in nature (atmospheric noise being used by random.org). Cryptographic hashes use a variety of hardware/software sources on a computer to generate better random numbers. Unity being based on .NET could make use of those functions instead, but I assume that there is a performance hit. At this point I assume that FGU just uses .NET's Random class (link provided in earlier post).

[Jiminimonka]

I'm gonna throw in my 2 cents (or pennies 'cos I am from London, init) - I don't know a damn thing about statistics or dice rolls etc., but I did remember back in the 1980s that RNG on computers was flawed and often came up with the same number a lot.

My players and I are constantly rolling crap numbers, consecutive misses on spells and attacks (lots of 1,2,3) or have streaks of 20 20 20 20... and I know that RNG is RNG etc. - but at least one of them is a genius at pattern recognition (not joking the dude is off the charts) and he and the other players think the RNG is off in FGU. So maybe adding some new random element to the code like pulling a seed from Random.org or whatever site Weissrolf suggests, might improve the RNG on dice rolls. That might stop players thinking the system is slightly borked.

Either way, I love FantasyGrounds and **** dice or good dice are part of the fun in D&D - just gets frustrating if its bad A LOT (and if its good a lot no one complains).

May the Smell of Neck be with you, always!

[Kelrugem]

huii, a lot of going in this thread, I am sorry, I cannot really read anything now, but I skimmed a bit:


"Uniform" in the headline, "random" in the text body. As I am no expert I might mix things up, but currently I don't get the impression that I do.

"Uniform distribution" is just one form of maaaany possible probability distributions. When something is not uniformly distributed, then it can be still a "random variable" :) (and there are a lot of possible other probability distributions: Another very important example is e.g. the Gauß distribution (also known as normal distribution) where things clutter more likely around the mean value when taking the limit to infinity (= "the number of "rolls" taken")) So, just in case, uniform is not a necessary condition for being whatever we call "random" :)

A system with just one ideal die is uniform though, so, when you just referred to that, then everything's okay :) (as said, I have completely no time to read the whole conversation here now :D)

[Weissrolf]

Damned wrote that he did not find "a definition that links random and uniform together", so I had to specifically answer that part and referred to a sequence of single random integer numbers being created, not a sum of multiple dice.

I repeatedly get the impression that the counter-argument for analysis of dice results is "anything can happen", but I disagree with this assessment. Every single die result can happen equally probable, which over time accumulates to uniform distribution of results over the whole range. Sequences of same numbers are possible and happen, but their rate is determined by probability and their distribution should not follow a (even hidden) pattern.

From there every gaming system makes different uses of dice being combined and random elements being weighted against fixed boni.

I mentioned Gussian/normal distribution in reference to white-noise amplitude, where both uniform and Gaussian distribution can be used to create different loudness profiles, while energy is still distributed over all frequency bands uniformly. Things get more complicated with bandpass vs. FFT interpretation of said distribution, but that's not important here.

Keep in mind that I am no expert in this field and not even a native English speaker. The latter being another difficulty on top of the usual intricacies of forum communication.

[LordEntrails]

My players and I are constantly rolling crap numbers, consecutive misses on spells and attacks (lots of 1,2,3) or have streaks of 20 20 20 20... and I know that RNG is RNG etc. - but at least one of them is a genius at pattern recognition (not joking the dude is off the charts) and he and the other players think the RNG is off in FGU. So maybe adding some new random element to the code like pulling a seed from Random.org or whatever site Weissrolf suggests, might improve the RNG on dice rolls. That might stop players thinking the system is slightly borked.
This comes up frequently from players. The GM has a file on their computer called chatlog.html. Every roll they have made is in there. Numerous times people have taken that log and evaluated it for randomness. Everytime it has proven to be within expected random distribution. Everytime.

Your player's minds are doing what is called confirmation bias, plus add in they probably don't understand randomness well anyway. They probably expect something more akin to a media players "shuffle" and not a "random" result.

For everyone else;
1. More than once people have wanted to know the technical details FG uses. The devs have never said more than what has already been mentioned in this thread. You can keep beating your head against a wall, but as some point other people are going to think you are stupid for doing so.
2. Getting testy with other posters and inferring they are more interested in defending FG than in finding some "truth" is not acceptable for the forums, and really is not having faith in the good nature of other posters. Everyone should be taking the input from others in good faith. Disagreement is fine, negative innuendo is not.
3. All of this is interesting, to varying degrees, but until someone can actually demonstrate an issue with the actual results of FG randomness, its pretty much an academic discussion. Let's keep that in mind and the importance that has to our hobby.

[Jiminimonka]

This comes up frequently from players. The GM has a file on their computer called chatlog.html. Every roll they have made is in there. Numerous times people have taken that log and evaluated it for randomness. Everytime it has proven to be within expected random distribution. Everytime.

Your player's minds are doing what is called confirmation bias, plus add in they probably don't understand randomness well anyway. They probably expect something more akin to a media players "shuffle" and not a "random" result.

For everyone else;
1. More than once people have wanted to know the technical details FG uses. The devs have never said more than what has already been mentioned in this thread. You can keep beating your head against a wall, but as some point other people are going to think you are stupid for doing so.
2. Getting testy with other posters and inferring they are more interested in defending FG than in finding some "truth" is not acceptable for the forums, and really is not having faith in the good nature of other posters. Everyone should be taking the input from others in good faith. Disagreement is fine, negative innuendo is not.
3. All of this is interesting, to varying degrees, but until someone can actually demonstrate an issue with the actual results of FG randomness, its pretty much an academic discussion. Let's keep that in mind and the importance that has to our hobby.

I think we need Lawrence Krauss to verify this. Some quantum mechanics should help.

[SilentRuin]

This comes up frequently from players. The GM has a file on their computer called chatlog.html. Every roll they have made is in there. Numerous times people have taken that log and evaluated it for randomness. Everytime it has proven to be within expected random distribution. Everytime.

Your player's minds are doing what is called confirmation bias, plus add in they probably don't understand randomness well anyway. They probably expect something more akin to a media players "shuffle" and not a "random" result.

For everyone else;
1. More than once people have wanted to know the technical details FG uses. The devs have never said more than what has already been mentioned in this thread. You can keep beating your head against a wall, but as some point other people are going to think you are stupid for doing so.
2. Getting testy with other posters and inferring they are more interested in defending FG than in finding some "truth" is not acceptable for the forums, and really is not having faith in the good nature of other posters. Everyone should be taking the input from others in good faith. Disagreement is fine, negative innuendo is not.
3. All of this is interesting, to varying degrees, but until someone can actually demonstrate an issue with the actual results of FG randomness, its pretty much an academic discussion. Let's keep that in mind and the importance that has to our hobby.

This is more psychological science than random science. People notice streaks of super bad or super good. They never notice or remember streaks of average. Hence why the chat logs are the place to go for any question on the random science part. There is no cure for the psychological part. At least for me. I only ever notice the good or the bad streaks.

[damned]

Consider this -

Roll 100 of each die (not percentile).
Almost 89% of results will be 10 or lower.
Over 80% will be 8 or lower.
Over 69% will be 6 or lower
Over 51% will be 4 or lower
and 1 in 8 rolls will be a 1

Now 100 of each dice is not a likely set of dice to roll, some dice roll far more and others far less
But no matter what set of dice you use, if they are of varying sizes you will get more results to the lower end of the range, substantially more in some cases

[Weissrolf]

I found a way to feed dice results into the ENT (Pseudorandom Number Sequence Test Program) software, but I preferably need to roll D256-1 (0-255) to get full bytes. So once the new version of MoreCore is available I will try to roll said custom dice and then test the results.

The biggest problem is converting text decimal numbers to binary data. I found a procedure, but the Windows clipboard still produces some errors along the way that I hope to circumvent with third party software. If anyone knows software that offers to directly convert said data, let me know.

Smaller rolls can be used with some caveats, like checking serial correlation and quantitatively comparing error results of FG vs. random.org (errors happen due to less than 256 values being used per byte).

[Weissrolf]

Here is a list of common problems with pseudorandom generator:


Potential problems with deterministic generators

In practice, the output from many common PRNGs exhibit artifacts that cause them to fail statistical pattern-detection tests. These include:

Shorter-than-expected periods for some seed states (such seed states may be called "weak" in this context);
Lack of uniformity of distribution for large quantities of generated numbers;
Correlation of successive values;
Poor dimensional distribution of the output sequence;
Distances between where certain values occur are distributed differently from those in a random sequence distribution.

Defects exhibited by flawed PRNGs range from unnoticeable (and unknown) to very obvious. An example was the RANDU random number algorithm used for decades on mainframe computers. It was seriously flawed, but its inadequacy went undetected for a very long time.

In many fields, research work prior to the 21st century that relied on random selection or on Monte Carlo simulations, or in other ways relied on PRNGs, were much less reliable than ideal as a result of using poor-quality PRNGs.[4] Even today, caution is sometimes required, as illustrated by the following warning in the International Encyclopedia of Statistical Science (2010).[5]

The list of widely used generators that should be discarded is much longer [than the list of good generators]. Do not trust blindly the software vendors. Check the default RNG of your favorite software and be ready to replace it if needed. This last recommendation has been made over and over again over the past 40 years. Perhaps amazingly, it remains as relevant today as it was 40 years ago.

[seycyrus]

***Pffft***

Atmospheric noise ... Looks like another piece of jargon that is being thrown around for pea**** purposes.

Yeah, electrical discharge ...yada yada yada.

How does random.org use atmospheric noise to generate their numbers? You don't know. I doubt you care.

Again, I'm not an employee of SW, not am I affiliated with the forums in any way except for being a user, just like yourself. So please no need at a pretense of being offended.

When this idiocy first started I said, you're gonna ask for 100 rolls, then 1,000, then 10,000, then 100,000, then a million, then 10 million ... The response I got was one of sarcastic indignation. But I was right.

Smite Works - Do not supply these guys with any more information. They are not worth the time. They are not dealing in good faith - pretending to be trying to find answers for the good of the community - what a farce.

They cannot find anything wrong with the results. Especially in light of how the results are used and applied in a gaming scenario.


... Since I am neither Mathematician nor Physicist I may get this wrong, though, would be nice if one could chime in here.

Thanks for the invitation. As a physicist, I will chime in here. You are wrong, on lots of things. Keep kicking up dust, but don't complain that you can't see.

No, SW doesn't have the burden of proof. Sorry, but just repeating it doesn't mean it's true.

[SilentRuin]

Like I said, this whole thread is psychology not science. Time to turn off the notifications on this one.

[Zygmunt Molotch]

To those with a genuine interest in testing random numbers, and pseudo random generators the software available from https://webhome.phy.duke.edu/~rgb/General/dieharder.php

is a go to tool

if you have the time, it's worth drilling down

while one user was correct you can never prove true randomness (not even pi), it is incorrect to say that one cannot test for pseudo-randomness

what we've learnt from the last two comments is that there are non physics professionals who are vitriolic enough to shut down critical questioning :)

your anger gets you no where, and isn't welcome

[damned]

My exact quote was:

Even with the seed you could not prove true randomness - you can only disprove it.

Which is not what you are suggesting I said.

[Weissrolf]

@seycyrus

First of all, you are mixing up responses here. I did not write about burden of proof, albeit I would very much prefer for SW to better explain their RNG method. On the contrary, I am spending my free time looking into these things and sharing results here. And the reason I was asking for rolls is because I am not able to easily provide them myself. As a physicist you should know that collecting data is the first step of analysis.

I have no horse in the race here. If FG's RNG works fine then all the better, if not then we need to find out and ask for a fix. I am not capable of running the more scrutinizing Diehard and TestU01, else I would do so and share the results. But I worked hard to get the simpler ENT results and now I need to ask for data again (or rather wait for MoreCore publication) to feed it.

All that being said, could you please be less condescending and rude? Thank you.

PS: We cannot make use of Dieharder or TESTU01, because both are said to require having the PRNG as part of the test-suite (need to be coded in). And since we have no access to the PRNG of FG we cannot do that. So the only qualitative "official" testing method available to us seems to be ENT.

[Zacchaeus]

Alright. I'm going to close this thread now before it descends into a flurry of insulting posts. Oh, wait.. too late for that.