Players are complaining about low rolls (they think it's a FG problem)

[GainunX]

Some members of my group are still complaining about low d20 rolls. The problem is that this complaint began to be addressed to the FG: they think it is a problem in the software. I particularly do not believe it, it seems to me too simple to program a dice roll with the correct probability for each result. But players seem to overvalue test failures and tend to become stressed. How can I explain to them that FG does not have to do with low rolls in a session?

[Lexfire]

Use the low rolls as part of the narrative, something of substance that can impact the course of the encounter, a skill check, the initiative sequence, or whatever roll the players are concerned about. Players have options - retreat from battle, defensive stand, support or assist another player. Perhaps something is learned or observed on a really poor roll, or information is available lower in the initiative count which may help the group the next round. I suspect there are fewer complaints when rolls are higher :)

[Zacchaeus]

This sort of post comes up now and then, but there is no evidence that there is anything wrong with the way FG rolls the dice. Note that the dice roller uses a physics engine and the number which is rolled is determined when the dice stop rolling. In other words it is just like real dice on the table; there is no random number generator involved.

Statistically of course you can roll low many times in a row; just as easily as you can roll high. Players notice when they roll low more often that they notice when they roll high.

[GainunX]

I think this explanation about physics engine should suffice. In fact, I believe it to be more of a psychological impression than any concrete reason to think there is a error on FG. In face-to-face tables with real data, for example, they think there is a problem with the dice, they exchange for anothe dice, and so on. I Just needed a clear answer about the dice roller engine to have something objective as an objection to to unnecessary complaints at the table. Thanks.

[JohnD]

This question comes up 2 or 3 times a calendar year.

Any kind of "evidence" would require a number of rolls to be statistically valid that your players would be rolling continually for days to generate. At the end of it all, they'll find the average roll on a d20 is somewhere between 9.75 and 10.25 over 100s of thousands of rolls.

A session's worth of bad luck is just that; a bit of bad luck.

I remember reading that FG doesn't use a basic random number generator, but also physics behind how the dice are rolled, the starting position of the dice, etc....

Point being there is no issue. However, if your players are unconvinced, have them do the research, but it won't just be a few sessions worth if a handful of rolls to gather.

There used to be a bit of a running gag in two of my C&C games that the party was cursed with terrible rolls. Over time it evened out but we still chuckle about it even a couple of years after the fact.

[dulux-oz]

If your players are complaining about too many low (or high) rolls then get them to provide some stats ie get them to record every die roll over 5-10 sessions, then total up each result (for each type of roll) and see if there is any significant difference from the "mathematically pure" expected results.

A PITA, I know, but I have not heard of anybody actually doing this who found a statistically significant error - but you need a large sample size (well over 1000 rolls) for the results to be statistically significant.

In other words, as hard as it is to prove a negative, there is just no proof that FG is rolling anything but "real" results :)

Cheers

[dulux-oz]

There used to be a bit of a running gag in two of my C&C games that the party was cursed with terrible rolls. Over time it evened out but we still chuckle about it even a couple of years after the fact.

We had a guy once who could roll exactly what he wanted (both physically and via FG) every time - except when the die roll *really* counted. Then he would always, always roll the worst result possible.

His name was (pseudonym - protecting the innocent(?)) John Burrows, so this phenomenon became know as the John Burrows Luck factor, or the JBL. Now, years later and with entirely different players, we still talk about being cursed with the JBL whenever something doesn't go our way :)

[damned]

Have those players turn on Manual rolls if they really are concerned.

[Doswelk]

Have those players turn on Manual rolls if they really are concerned.
If I did that then all my players would 17-20 every roll!

Note it must be stated that a) we do not game face-to-face and b) it will be a while before I run a D20 game! (But it will happen as I have bought the entire Starfinder range)

[celestian]

I try and point out when they have high rolls so they don't forget those ... Players tend to only remember the bad rolls ;) My players also like to complain about npcs rolling better but then I try and remind them I almost always have more bad guys than party size so... yeah, I get more rolls.

[LordEntrails]

At one point Doug setup a campaign stats page for a handful of campaigns. It showed things like who did the most damage, crits, etc. I think it also included average roll info too. Not sure what to search, but you might be able to find some of the posts that have links to these pages.

All the ones I looked at showed what appeared to be everything withing expected deviations.

[ddavison]

Yep, we have a chat parser we can upload to that will show the frequency of all rolls and it also tallies who hits and misses the most and lots of other stuff. Distribution on the # of campaigns we processed seems to all fall within normal bounds.

I'm guessing that if you asked them if the DM's rolls are also low they would say "no". :)

[Bonkon]

I think this explanation about physics engine should suffice. In fact, I believe it to be more of a psychological impression than any concrete reason to think there is a error on FG. In face-to-face tables with real data, for example, they think there is a problem with the dice, they exchange for anothe dice, and so on. I Just needed a clear answer about the dice roller engine to have something objective as an objection to to unnecessary complaints at the table. Thanks.

Good Day GainunX :)
When we used to play face to face everyone always had numerous dice that they could change out to if the rolls were bad, here our players change the die color for the same psychological effect :)

[Erin Righ]

It's called a confirmation bias, and it is something that scientists are constantly on the look out for. I basically goes; before my accident there were not as many amputees in my home city. The statement is patently absurd on its face, but now that I am an amputee I see amputees everywhere, what gives? Confirmation Bias. When looking for something you are more likely to see what you are expecting. Ask people what percentage of GDP the US spends on foreign aid, and many Americans asked will respond with the numbers of 10-25%, then you tell then that the US spends like 3-4% on Military and they will change there estimation to high single digits (7-9%). These people expect the government to spend highly on foreign aid, and will even (at times) say they can cite non-existent sources putting foreign aid so high. This is the same thing, they expect it, so they believe they saw it. If you are at all interested, I believe the US spends about 0.1-0.2% on foreign aid, but you can look that up. What I do know is that the military budget is over 10 times the foreign aid budget. Dulux-oz is correct, you can have them log rolls, but you would need thousands of rolls to show averages or you can use Celestian's advice and don't let them forget the good stuff, make critical success just as memorable as critical failure. They'll get over it.

[Erin Righ]

I should say that the statistics for GDP and foreign aid are similar in most developed countries, I am in no way dumping on Americans here.

[Nylanfs]

Tell them yo change the different colour of dice, yea placebo effect!

[Nylanfs]

Or you buy a white car, suddenly there are white cars everywhere.

[Erin Righ]

Or you buy a white car, suddenly there are white cars everywhere.

Exactly, I never saw another Subaru Forester until I bought one for my wife, now you'd think every middle aged housewife in BC drove one from the amount I see them, and what's up with them all being driven by middle aged housewives, anyway?

[LordEntrails]

Exactly, I never saw another Subaru Forester until I bought one for my wife, now you'd think every middle aged housewife in BC drove one from the amount I see them, and what's up with them all being driven by middle aged housewives, anyway?
Thanks, I needed an amusing chuckle :)

[Ken L]

Players will always find something to blame when the rolls don't go their way. They'll get wiser the more they play, or you'll get more tired of their BS, one or the other. ;)

[Erin Righ]

Well I have to say, my dice favor my players, if I ever run a game, play, I actually have to fudge rolls to make things harder for my players, if I don't they don't feel that I am challenging with the amount my dice fumble in their favor

[PTBBC.ORG]

The only thing we ever see that my players talk about with dice rolls is the same number coming up several times in a row. Last night I rolled 3 D20's for checks and got 3 8's in a row. We see that from time to time, but we don't complain. Just laugh and roll with it.

[Donrav]

Seems that when you roll the dice manually into the box it's all fine. Hence the reason most of the investigations seem to have the proper outcome.

However; I did notice that each session at least 1 player has the "lucky rolls" and at least someone has the "bad rolls" when using the roll clicking from your character sheet for combat actions.
It changes each session who it is and who is affected, but rolling 4 times a 16 in a row, and almost only 17, 18 and 19 on fight related rolls by the DM or any player stands out and can happen once, but is very unlikely to happen multiple times across only a few sessions.
So with the physics in mind, it could be that the starting point of the dice is determined per session per character sheet only for character sheet based rolls and thus effects the overall outcome per session.
This will be very hard to prove as you would need to record behavior across multiple sessions with multiple characters only on the fight related rolls.

However, we do seem to be able to determine at the start of a session who has the lucky rolls and who doesn't.
And that is very suspicious of some code interfering with the actual supposed randomness to me.

So I'm very biased on the actual coding precision in FG when relating to the character sheet actions.
i would love to see the actual programming code of the dice behind the character sheets at some point.

[amerigoV]

I suggest a different approach. Accuse your players of using loaded dice when they play in person. Make them provide their real dice are not biased.

(tongue in cheek, or course)

[tahl_liadon]

...I suspect there are fewer complaints when rolls are higher :)

X-D true. true.

to gm with complaining players -- tell them there are 3 proven scientific methods to improve rolls:

1) change dice color at least 3 times during a session; best result: immediate after a roll of 1
2) "flush the dice" -- set d20 to roll *at least* 20x in a single roll in chat window; best result: when d20s is set to roll all at once and fill the entire chat window
3) (this one works every time) say out loud "20" whenever player wants a crit

works for me.

[LordEntrails]

Their is a 'chatlog.html' file saved inside the campaign folder for each campaign. You can use the chatlog scrubber (found on the forums via a search) and modify it to pull out all the rolls and do statistical analysis on the rolls.

Then you actually need to go study statistical analysis and distribution. What you think you are seeing you will learnis within statistical norms (see the results of your statistical analysis of your actual rolls) and has to do with the human psyche and the fact that humans tend to take note of those things they think prove their point and ignore those facts that disprove their thesis. i.e. One player comments that they always get low rolls, so everyone notices the low rolls they make and ignore the high rolls they make.

[Speculi]

I extracted every D20 roll once from the chatlog in a D&D 5e campaign, counting also the dropped dice of advantage/disadvantage. By now I should have even more data, but I don't see a point in going through this again. The theory of per session bias can't be addressed with this method, so maybe that's something one could look into... but in general the probabilities seem to be right.

2231 rolled D20 in total. Each value rolled with relative amounts in a range of 4.08% to 6.01% seems reasonable close to the expected 5%.

Here are the complete results.
https://docs.google.com/spreadsheets/d/1-Af2a-X5OwSgRVEwdbE_xRgGDiWhpEMd2kXWDofCnWE

[Bonkon]

Good Day All :)
Last nights session was a classic example of a bad streak of rolls. Our Monk (5E) was attacking and rolled three consecutive 1's in the same turn. To make matters worse our DM is using the Crit/Fumble tables :) The player is convinced the DM has a way to force a fumble!! LOL :)

[Mgrancey]

I have either co-players or players (forget which campaign) first they they do after logging in is to roll something like 20d20s to see what they spread is and if they need to change the color. Personally, I have found that changing the color and the text of the dice so I can't see what i actually roll is reported seems to benefit me (so white dice on white background OR black on black), mind you this ISN'T something I have scientifically tracked, just something that I have run thru my head.

[JohnD]

When in doubt always change the dice color... scientifically proven to make a difference.

[Zacchaeus]

Good Day All :)
Last nights session was a classic example of a bad streak of rolls. Our Monk (5E) was attacking and rolled three consecutive 1's in the same turn. To make matters worse our DM is using the Crit/Fumble tables :) The player is convinced the DM has a way to force a fumble!! LOL :)

Last session we had the Monk character got 3 natural 20's in a row, so there must be something fishy going on. :)

[Bonkon]

Last session we had the Monk character got 3 natural 20's in a row, so there must be something fishy going on. :)

Good Day Zacchaeus :)
I will have to pass this on to our Monk, perhaps he will blame your Monk for taking the 20's from him!! :)

[Ulric]

In my Starfinder campaign I have a player with a Vesk Soldier. His rolls are consistently so low he thinks FG hate him. In all seriousness it is sad when a player rolls badly so frequently that it begins to frustrate them.

[Weissrolf]

How does FG determine random numbers, aka what is the seed and how trustworthy are consecutive rolls? Will there be a difference between Unity and Classic algorithms?

My personal observation is that dice roller programs have a tendency to repeat consecutive numbers, so this is the first thing I test with any dice roller. This become especially apparent with Backgammon programs that tend to roll more doublets than I have ever seen to happen at a real table.

[Nylanfs]

AFAIK the seed is generated by how fast and the direction you are moving the mouse, and it uses a physics engine to generate the roll.

[Weissrolf]

But we mostly use double-clicks to roll. So no mouse-movement is happening. The dice of one of my players always spins like a gyroscope instead of rolling. The other players‘ dice do roll, though.

[Kolimar]

I have players who are complain for the same and we play...the dark eye's system. Players are always players...

[damned]

But we mostly use double-clicks to roll. So no mouse-movement is happening. The dice of one of my players always spins like a gyroscope instead of rolling. The other players‘ dice do roll, though.

alt....

[Nylanfs]

NOOOO, Don't tell the secret! :)

[Weissrolf]

alt....

Thank you, this explains it. This is a problem of him having to use a remote desktop connection to my NAS' virtual Windows machine which is running the FG client. He is using Linux and does not like having to run Wine, so the remote connection seems to fire the ALT key for some odd reasons.

So where does the thrust/spin for double-clicks come from (aka what is the seed then)?

[Nylanfs]

I'm still fairly sure it uses your mouse movement as you click the button, but John would have to chime in here.

[LordEntrails]

You can always analyze your chatlog.html for roll distribution. It has been done before and always falls within expected statistical normals for random behavior.

[Mytherus]

I think this notion that fg messes up the dice is comical. My players and I play a laid back style so we joke about it..no one really thinks fgs dice roller is messed up though.

But thats not the point of my post...i wanted to share on the flip side of an unlucky streak recently (sat 4/25..our bgdia game) i had two players in sequential initiative order roll two crits EACH back to back AND the each rolled max possible damage each time too. It was uncanny and we had a lot of fun with that.

[Honken]

I have a really, really, really good solution on this problem. Find a game system where you have to roll low. GURPS comes to mind, CoC, and if I am not mistaken the new (for FG) and fancy Conan 2d20 system.

/H

[gaara6666]

Yea h ive used fg for 11 years. You can get a lot of the good or the bad in a row. It is just random. But when my dice arent lucky i always tell people "oh you just need to change the die color" and that has been the ongoing joke for years.

[LordEntrails]

Yea h ive used fg for 11 years. You can get a lot of the good or the bad in a row. It is just random. But when my dice arent lucky i always tell people "oh you just need to change the die color" and that has been the ongoing joke for years.
Much less expensive then smashing them :)

[JohnD]

Much less expensive then smashing them :)

Not as satisfying though.

[Weissrolf]

I'm still fairly sure it uses your mouse movement as you click the button, but John would have to chime in here.
When you click a button there usually is no mouse-movement, though.

[Bilbobolen]

Really, FGU uses a "physics dice generator" rather than simply using a rnd function in whatever language it is coded in. So the rolling dice I see on my screen actually mean something as they roll. From a programing point of view that seems excessively complex and prone to biases. I assume it has been tested but still.

[LordEntrails]

Really, FGU uses a "physics dice generator" rather than simply using a rnd function in whatever language it is coded in. So the rolling dice I see on my screen actually mean something as they roll. From a programing point of view that seems excessively complex and prone to biases. I assume it has been tested but still.
As I said, you can go roll a million dice and then parse the chatlog.html and view the distribution. It will show the results are within statistical norms.

Is it excessively complex? Sure, but it's one of the things that make FG feel the way it does.

[Pathf1nder]

Yeah I just started to use FG unity and I cant seem to roll over 10 90% of the time and where it gets real bad is when you have advantage and roll both under 5 fairly consistently. I'm playing some of the solo modules
and is so frustrating to keep missing. I just use the double click to attack.

[damned]

Please share your chat log.

[LordEntrails]

Yeah I just started to use FG unity and I cant seem to roll over 10 90% of the time and where it gets real bad is when you have advantage and roll both under 5 fairly consistently. I'm playing some of the solo modules
and is so frustrating to keep missing. I just use the double click to attack.
In your FG Data folder their is a campaign folder, in there is a folder with the name of your campaign. Please open it, and copy the chatlog.html out of there and post on the forums.

[PaganBlack]

I would be happy to upload the chat. I'm getting to the point to where I'm telling my players to re-roll. The randomizer is not very random. I'm hoping this will allow the developers to better improve the randomizer. We really enjoy FGU and love the product. keep up the good work =)

[damned]

the first 100 d20s resulted in:
6 8 14 15 5 6 4 15 2 10 18 20 12 6 14 12 2 5 11 1 8 3 2 10 11 11 14 19 9 9 17 2 5 5 18 2 9 5 2 8 13 15 12 17 13 5 5 9 1 3 18 19 13 16 10 6 1 18 20 20 18 19 20 2 8 13 5 13 15 7 4 11 1 9 6 5 2 15 4 16 20 6 19 7 9 19 7 13 11 5 13 13 7 7 17 2 18 5 9 9
with an average of 9.98
4 1s
5 20s

For the d4s in that same time eg - until I counted 100 d20s
9 d4s rolled
average 2.333
3 1s
1 4s

For the d6s in that same time eg - until I counted 100 d20s
10 d6s rolled
average 3.5555
0 1s
1 6s

For the d8s in that same time eg - until I counted 100 d20s
14 d8s rolled
average 4.6428
2 1s
2 8s

What I posted is ofc only a small sample statistically but I didnt see any long runs of numbers or abnormal numbers of 1s
the d20 average is slightly down but the sample size is small

16 and 2 is typically rolled above average in all the big sets of d20 rolls we have done
16 only had 2 matches
but 2 had 9 matches

I dont have any good way to auto parse all the data - i used an expression to delete all data that wasnt between [ ]
and then manually pulled the roll results (not the totals) until I got 100 d20 results

Do those numbers look way off to you?
To me they look ok.

thank you for posting your chat log

[PaganBlack]

That is a years worth of chat. try the last 100 d20 rolls. ;-)

[damned]

I will look tomorrow - its 0120am here :bandit:

[PaganBlack]

OMG go to bed. =) no rush....

[Weissrolf]

Edit: wrong calculation!!! Fix will come in a minute!

[Weissrolf]

Fantasy Ground does not roll random with uniform distribution, especially not with click-/automation-rolled 3D animated dice. If you want better randomness then you need to use the modded Instant Dice extension. It is still not very good, but considerably better than 3D dice.

https://www.fantasygrounds.com/forums/showthread.php?62047-Instant-Dice-Disable-Rolling-Animation&p=561529&viewfull=1#post561529

In your chatlog the distribution of D20 results is as follow:

https://i.imgur.com/wrlMEb2.png

For comparison: You rolled 18% more 20s (124) than 1s (105), out of a total of 2334 rolls. Like Damned wrote, the number of rolls from a chatlog is rather small for statistical analysis, but if you want better randomness than eye candy then use the linked extension.

[PaganBlack]

I checked out that extension, MUCH better with the randomness.

[Weissrolf]

Make sure to use the "modded" version, else it will roll the very same list of results every time you restart FG.

FG Classic's 3D click-rolls also seem better than FG Unitiy's. Problem with FGU is that you cannot change dice if you don't like a specific die.

In the following article they think that a standard deviation of 0.33% from the expected 5% is okish for 10k rolls (85% of all rolls should be within 0.33% of the expected 5%). Over 1 mio. D20 rolls I calculated that FGU has a standard deviation of 0.25%, FGC 0.14% and Instant Dice 0.02% (more rolls = less deviation expected). I would have to recalculate for 10k rolls for better comparison.

https://www.awesomedice.com/blogs/news/d20-dice-randomness-test-chessex-vs-gamescience

[Weissrolf]

I did the calculation for the first 10k rolls of my 1 mio. samples. Standard deviation of FGU is 0.34%, FGC 0.32% and Instand Dice 0.18%. This helps to better compare the results to the article about real dice rolls (+-33 rolls in the article = +-0.33%). Single results are more off than this, so this is kind of an average.

[Jiminimonka]

Some members of my group are still complaining about low d20 rolls. The problem is that this complaint began to be addressed to the FG: they think it is a problem in the software. I particularly do not believe it, it seems to me too simple to program a dice roll with the correct probability for each result. But players seem to overvalue test failures and tend to become stressed. How can I explain to them that FG does not have to do with low rolls in a session?

Do they ever complain when they get high rolls? Never is my guess.

This is just a perception issue.

[Jiminimonka]

Please Weissrolf don't hijack this thread with pages of charts and revive your other thread which has been addressed.

Dammed is going to look at the chat log instead.

[Weissrolf]

My chart is from the chatlog posted in this very thread. What hard evidence did you contribute?

[Jiminimonka]

My chart is from the chatlog posted in this very thread. What hard evidence did you contribute?

This is not about election fraud. I don't need hard evidence.

[Weissrolf]

And we don't need people trying to mob other users out of threads. Just because you don't like my view does not mean you are entitled to keep me and my line of discussion out of threads on this very topic. Some people prefer to discuss topic, others prefer to discuss people. I prefer the former, so this will be my last post on the latter.

[Ecks]

I parsed the d20 results from @PaganBlack's chat log and used Pearson's chi-squared (https://en.wikipedia.org/wiki/Pearson%27s_chi-squared_test) test on the results. This test can be used to evaluate how likely it is that the observed results arose by chance. There is a post here (https://rpg.stackexchange.com/questions/70802/how-can-i-test-whether-a-die-is-fair)that goes into more detail on using this test to check if a die is fair.

I did a fairly quick parse of @PaganBlack's chat log (a few results might have been dropped) and calculated the χ² test statistic. One thing to note is that it looks like this chat log includes results for both FGC and FGU.

Roll Observed Expected O-E (O-E)^2 (O-E)^2/E
--------------------------------------------------------------------------
1 250 248.8 1.2 1.44 0.006
2 237 248.8 -11.8 139.24 0.560
3 264 248.8 15.2 231.04 0.929
4 254 248.8 5.2 27.04 0.109
5 233 248.8 -15.8 249.64 1.003
6 238 248.8 -10.8 116.64 0.469
7 226 248.8 -22.8 519.84 2.089
8 259 248.8 10.2 104.04 0.418
9 238 248.8 -10.8 116.64 0.469
10 253 248.8 4.2 17.64 0.071
11 222 248.8 -26.8 718.24 2.887
12 245 248.8 -3.8 14.44 0.058
13 261 248.8 12.2 148.84 0.598
14 272 248.8 23.2 538.24 2.163
15 261 248.8 12.2 148.84 0.598
16 261 248.8 12.2 148.84 0.598
17 244 248.8 -4.8 23.04 0.093
18 231 248.8 -17.8 316.84 1.273
19 265 248.8 16.2 262.44 1.055
20 262 248.8 13.2 174.24 0.700
--------------------------------------------------------------------------
Total 4976 χ2 = 16.146

To determine a reasonable value of χ2, you can look it up in a table:

Probability less than the critical value
ν 0.90 0.95 0.975 0.99 0.999
----------------------------------------------------------
19 27.204 30.144 32.852 36.191 43.820

The null hypothesis is that the die is unbiased. At a 95% significance level, the critical value is 30.144. As the chi-squared statistic does not exceed it (χ2 = 16.146), we fail to reject the null hypothesis and conclude that there is insufficient evidence to show that the die is biased at a 95% significance level.

From my own campaign's chat log, I have 349 d20 rolls since switching to FGU (prior rolls were all FGC, so I excluded them):


Roll Observed Expected O-E (O-E)^2 (O-E)^2/E
1 12 17.45 -5.45 29.7025 1.702
2 27 17.45 9.55 91.2025 5.227
3 19 17.45 1.55 2.4025 0.138
4 18 17.45 0.55 0.3025 0.017
5 9 17.45 -8.45 71.4025 4.092
6 19 17.45 1.55 2.4025 0.138
7 20 17.45 2.55 6.5025 0.373
8 12 17.45 -5.45 29.7025 1.702
9 14 17.45 -3.45 11.9025 0.682
10 24 17.45 6.55 42.9025 2.459
11 21 17.45 3.55 12.6025 0.722
12 23 17.45 5.55 30.8025 1.765
13 15 17.45 -2.45 6.0025 0.344
14 15 17.45 -2.45 6.0025 0.344
15 11 17.45 -6.45 41.6025 2.384
16 21 17.45 3.55 12.6025 0.722
17 20 17.45 2.55 6.5025 0.373
18 18 17.45 0.55 0.3025 0.017
19 15 17.45 -2.45 6.0025 0.344
20 16 17.45 -1.45 2.1025 0.120
Total 349 χ2 = 23.665

The χ2 result of 23.664 does not exceed the critical value at 95% significance level, and so again there is insufficient evidence to show that the die is biased.

For a slightly contrived test, here are results with a d20 set to a hotkey and held down continuously to generate a bunch of rolls. I then parsed the results from the chat log:


Roll Observed Expected O-E (O-E)^2 (O-E)^2/E
1 23 37.85 -14.85 220.5225 5.826
2 42 37.85 4.15 17.2225 0.455
3 50 37.85 12.15 147.6225 3.900
4 35 37.85 -2.85 8.1225 0.215
5 38 37.85 0.15 0.0225 0.001
6 40 37.85 2.15 4.6225 0.122
7 28 37.85 -9.85 97.0225 2.563
8 39 37.85 1.15 1.3225 0.035
9 32 37.85 -5.85 34.2225 0.904
10 36 37.85 -1.85 3.4225 0.090
11 33 37.85 -4.85 23.5225 0.621
12 29 37.85 -8.85 78.3225 2.069
13 38 37.85 0.15 0.0225 0.001
14 37 37.85 -0.85 0.7225 0.019
15 34 37.85 -3.85 14.8225 0.392
16 62 37.85 24.15 583.2225 15.409
17 44 37.85 6.15 37.8225 0.999
18 54 37.85 16.15 260.8225 6.891
19 37 37.85 -0.85 0.7225 0.019
20 26 37.85 -11.85 140.4225 3.710
Total 757 χ2 = 44.242

At 99.9% significance level, the critical value is 43.82. χ2 = 44.242 exceeds the critical value - so we reject the null hypothesis (that the die is fair) and conclude that the die is biased at the 99.9% significance level. Note though, that these die results were generated only by hotkey (no direction/velocity input to physics engine) and over a very short period of time (in case time is used as an input to the physics engine).

TL;DR - from the chat logs of actual gameplay, there was insufficient evidence to show that the d20 results were biased. However, with contrived hotkey-generated results over a short period of time, the d20 is very likely biased.

[Weissrolf]

Nice analysis. How did you extract the D20 results from the chat-log and what software did the calculation for you? I seem to have done it wrong, because I only got 2334 results.

PS: "Short period of time" makes no difference according to my tests. I did many rolls (like 3x 1 mio. D20), with different pauses and the distribution followed more or less the same pattern.

[Ecks]

How did you extract the D20 results from the chat-log and what software did the calculation
I just used vim (a text editor) to filter for lines containing 'd20' - then used some text manipulation to strip out the XML to just leave "d20 + Modifier = Result". (I did note that some results are formatted as "[1d20]" and others as "[d20]" - I'm guessing this difference is from FGC versus FGU). I then used Excel to calculate d20 = Result - Modifier, and used Excel for calculating the tables and χ2.

[Weissrolf]

The D20 vs. 1D20 likely was the reason why I got less results then.

[Trenloe]

TL;DR - from the chat logs of actual gameplay, there was insufficient evidence to show that the d20 results were biased. However, with contrived hotkey-generated results over a short period of time, the d20 is very likely biased.
Thanks for the summary. This show that actual game play results are OK, whereas a contrived artificial simulation is not. Thanks for the work and the clarification.

[Weissrolf]

First of all, the results include FG Classic dice, which in my tests have demonstrated more uniform distribution. If I find time I may do the same calculation with only the FG Unity results.

Then:
If χ2≤27, the die probably has no bias, or at least you haven't counted enough rolls to detect it.
Statistical analysis needs a minimum (often rather higher than lower) amount of data to make meaningful statements, that's the curse of probability.

[Marquis_de_Taigeis]

im wondering if anyone has thought about creating an extension, which would during a session keep track of all the die rolls for each person

Ive always found that if anyone has felt they are having a particular bad day rolling dice both physically and virtually, the perception of how well someone is rolling can be easily changed with the swapping of dice(physical) / dice colour (virtual), and its very easy to only remember when you get a bad roll (as it normally affects the situation more detrimentally and often opens the story) than that of a good roll (which doesnt alter the situation from what you were hoping)

personally i love and remember when the dice roll poorly as i find it normally makes the story much more entertaining especially with death saves in 5E where it adds more tension if your sitting with 2 failed saves and then manage to make the next 3 / be healed its a lot more fun than no tension when your sitting with passed saves

[damned]

I went to an arbitrary point back from the end to try and ensure 100 d20 results
For expediency I only collected d20 results
I ended up with 172 results and they are:

19, 5, 19, 8, 19, 18, 6, 5, 17, 12, 14, 11, 15, 12, 14, 4, 20, 17, 18, 1, 6, 18, 11, 18, 17, 3, 3, 6, 18, 9, 11, 2, 10, 5, 20, 9, 5, 5, 5, 12, 14, 13, 16, 2, 8, 12, 18, 16, 14, 13, 19, 4, 4, 3, 20, 15, 10, 16, 14, 15, 2, 11, 15, 3, 20, 8, 13, 3, 5, 13, 19, 15, 16, 12, 11, 4, 1, 4, 20, 18, 14, 17, 3, 10, 15, 8, 6, 17, 10, 10, 15, 3, 19, 15, 6, 14, 14, 14, 18, 5, 15, 3, 12, 3, 15, 18, 16, 12, 8, 20, 19, 17, 5, 12, 19, 20, 14, 11, 9, 4, 10, 16, 8, 6, 5, 18, 17, 1, 12, 19, 7, 13, 11, 12, 3, 14, 3, 4, 5, 8, 14, 13, 16, 15, 20, 11, 9, 5, 15, 12, 20, 3, 12, 2, 15, 19, 5, 13, 9, 12, 13, 6, 20, 1, 5, 4, 16, 6, 8, 6, 4, 6

The average is: 11.151
There were 4 [1]s
There were 4 [2]s
There were 8 [16]s
There were 10 [20]s

Again, small sample size used, done manually, might have missed a roll or two or made an error in subtracting the modifiers from the result etc

Do those results match your groups perception?

[damned]

And I missed 3 pages of replies in between the post I was replying to and mine!

[Weissrolf]

At this point I suspect that whatever skews results in consecutive rolls of a single die (likely physics) is somewhat blurred away by rolling different dice. So when 1000 consecutive D20 can be proven to not be random then 1000 rolls of mixed dice (D20, D6, D100, etc.) cannot (as easily), as least not with a single test. In practice this might be a good thing.

[Skellan]

im wondering if anyone has thought about creating an extension, which would during a session keep track of all the die rolls for each person



There is a roll statistics extension for Savage worlds.

[Weissrolf]

Western Kentucky University
TopSCHOLAR®
Student Research Conference Select Presentations Student Research Conference

Spring 2019
Dice Mythbusters
C. Warren Campbell
Western Kentucky University, [email protected]
William P. Dolan
Western Kentucky University and Carol Martin Gatton Academy of Mathematics and Science,
[email protected]

https://digitalcommons.wku.edu/cgi/viewcontent.cgi?article=1048&context=sel_pres


Conclusions
...
6. Rolling D20 dice 100 times and using the obtained chi square values to judge relative fairness is statistically unjustifiable.
7. A D20 die must be highly unfair to be detected in a few hundred rolls.

[Weissrolf]

Sample size (whole table)
A sample with a sufficiently large size is assumed. If a chi squared test is conducted on a sample with a smaller size, then the chi squared test will yield an inaccurate inference. The researcher, by using chi squared test on small samples, might end up committing a Type II error.

Here are the results of large sample size tests (1 mio. D20).

Instant Dice (ID) extension (math.random):

16,23 = Chi-Square Statistic (df=19)
0,6419 = p-value (to 4 decimal places)
Conclusion: Within expectaction for random results at a Type I error rate of 0,05

FG Classic 3D click-rolls

714,51 = Chi-Square Statistic (df=19)
<0.0001 = p-value (to 4 decimal places)
Conclusion: Not equally distributed, some probabilities differ.

FG Unity 3D click-rolls

2337,54 = Chi-Square Statistic (df=19)
<0.0001 = p-value (to 4 decimal places)
Conclusion: Not equally distributed, some probabilities differ.

Keep in mind that this is a rather simple test that only looks at overall distribution. Consecutive roll results of 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20 ,1,2,... would not fail, because each result is equally distributed, still very obviously not random. So failure (3D) tell us more than success (ID).

[Castellanox7]

Welp, time to make an extension like the roulette "hot/cold numbers" so the saps...er... players can fall for the "dependent statistics" fallacy... >;)

[Weissrolf]

I just got a bad reputation for supposedly hijacking this thread. So I leave biased interpretation of results to others and leave you be.

[Moon Wizard]

Closing thread. This is an old thread; and the conversation went way off course.

Please note: small data sets are not good samples (as they will have higher variance); and in reality each RPG session on FG is only a small data set of rolls.

Regards,
JPG