Poker Mavens Statistics Question (1 Viewer)

[BearMetal]

Edit: So, the huge discrepancy was from analysis error (taking into account table hands, not player hands). In this post, I update my analysis with raw # of hands. It's still a little high ... all of my site's pocket pairs are happening too often. But not 5x too often. Instead, there's typically a 10-30 hand average discrepancy which is about 10-15% more often than they should happen. Perhaps this is just within the realm of randomness...

Hands	Occurrences	1-in-x Hands
Total: 156351
AA	827	189
KK	772	203
QQ	811	193
JJ	803	195
TT	799	196
99	820	191
88	783	200
77	777	201
66	826	189
55	739	212
44	826	189
33	805	194
22	830	188

In any event, please understand that I just opened this thread for feedback in help understanding what I saw. I never meant to point fingers, blame the software, etc. Just wanted to chat about it I've actually been a huge proponent of the fairness of PM. It's certainly light years more realistic than some of the online sites we tried.

OP (Used incorrect table hands vs correct player hands):

Spoiler: Original Wrong Analysis

I'm the last person here you would expect to post this. I've looked over the algorithm that Kent says shuffles the cards and it's random and not biased. I don't honestly believe that Poker Mavens has any reason to produce certain hands more often than others. Other online sites, who want to profit by gaining new players, have a reason to manipulate the cards to make hands way more interesting. PM already has our license money (a 1-time fee); there's no reason to produce manufactured hands.

But here's the thing, I've noticed too many things that shouldn't happen on my PM site happening too often.

For example, we've seen 33,662 hands and out of those, we've had 6 Royal Flushes and 13 Straight Flushes. That's more than double what each should be. It's hard to break down this information since this only represents the hands that were played/shown; I'm sure there were more RF's and SF's that were folded to.

So, I did a much simpler test: I took a look at how often pocket pairs come up. Turns out, AA came up nearly 5x as often as it should. A pocket pair should appear once every 221 hands. Out of 33,662 hands, that means it should appear about 152x. I ran this query for all pocket pairs, and it turns out they are all 5x as often:

AA​	803​
KK​	741​
QQ​	791​
JJ​	778​
TT​	773​
99​	802​
88​	755​
77​	748​
66​	801​
55​	716​
44​	800​
33​	792​
22​	818​

If you add them all up, you get a total of 10,118 pocket pairs over 33,662 hands which is roughly 1 out of every 3. I typically have about 14 players on average, so if you say that every 3 hands, 1 out of my 14 players gets a pocket pair, that would mean that roughly each player gets a pocket pair every 42 hands. Again, this is roughly 5x as often as they should!

In case you wanted to run these queries on your own server:

# total hands dealt:
egrep -e 'Seat.*\[\w\w \w\w\]' HandHistory/* | wc -l | sed -e 's/^[[:space:]]*//'

# AA through 22
egrep -e 'Seat.*\[A\w A\w\]' HandHistory/* | wc -l | sed -e 's/^[[:space:]]*//'
egrep -e 'Seat.*\[K\w K\w\]' HandHistory/* | wc -l | sed -e 's/^[[:space:]]*//'
egrep -e 'Seat.*\[Q\w Q\w\]' HandHistory/* | wc -l | sed -e 's/^[[:space:]]*//'
egrep -e 'Seat.*\[J\w J\w\]' HandHistory/* | wc -l | sed -e 's/^[[:space:]]*//'
egrep -e 'Seat.*\[T\w T\w\]' HandHistory/* | wc -l | sed -e 's/^[[:space:]]*//'
egrep -e 'Seat.*\[9\w 9\w\]' HandHistory/* | wc -l | sed -e 's/^[[:space:]]*//'
egrep -e 'Seat.*\[8\w 8\w\]' HandHistory/* | wc -l | sed -e 's/^[[:space:]]*//'
egrep -e 'Seat.*\[7\w 7\w\]' HandHistory/* | wc -l | sed -e 's/^[[:space:]]*//'
egrep -e 'Seat.*\[6\w 6\w\]' HandHistory/* | wc -l | sed -e 's/^[[:space:]]*//'
egrep -e 'Seat.*\[5\w 5\w\]' HandHistory/* | wc -l | sed -e 's/^[[:space:]]*//'
egrep -e 'Seat.*\[4\w 4\w\]' HandHistory/* | wc -l | sed -e 's/^[[:space:]]*//'
egrep -e 'Seat.*\[3\w 3\w\]' HandHistory/* | wc -l | sed -e 's/^[[:space:]]*//'
egrep -e 'Seat.*\[2\w 2\w\]' HandHistory/* | wc -l | sed -e 's/^[[:space:]]*//'

Edit: So I seems like just going for "raw" hands player instead of table hands was what was throwing off my stats.
https://www.pokerchipforum.com/threads/poker-mavens-statistics-seem-off.69604/post-1409019

 

[Senzrock]

@Finest

 

[Senzrock]

And this isn't a case of pocket pairs, especially AA being strongly skewed because they *are shown down* vs many other types of hands?

 

[waddadonk]

Edit: upon rereading, nevermind

Double edit: maybe this stackoverflow is right? Cannot confirm.

https://poker.stackexchange.com/a/1206

"It's difficult to answer your question as to why the probability "seems too high", since the numbers won't seem high to someone who lives and breathes probabilities. But perhaps this will help: At a 10 person table you're getting 10 tries at a 1/17 shot. The chance of at least 1 person getting a pocket pair is 45%. So maybe now it's easier to see that the likelihood of 2 or more getting a pocket pair is less than 45%, but not "much" less."

 

[Highli99]

Are you basing the primary assumption in 33k hands played or total number of hand combos. I'm assuming you should base on total hand combos (ie 9 players times 33k hands)

 

[madforpancakes]

So, I did a much simpler test: I took a look at how often pocket pairs come up. Turns out, AA came up nearly 5x as often as it should. A pocket pair should appear once every 221 hands. Out of 33,662 hands, that means it should appear about 152x. I ran this query for all pocket pairs, and it turns out they are all 5x as often

This math doesn't add up to me. A specific pocket pair should appear once in every 221 hands per player. More players at the table = more pocket pairs. You should either look at a single players dealt hand history or figure out how many players are at the table at each point, which seems way more difficult to do with just some greps.

 

[Finest]

A specific pocket pair (like AA) should only appear every 221 hands, but any pocket pair should appear once every 17 hands (3/51).

Also, when you say you've seen 33,662 hands, do you mean that 33,662 hands have been dealt on your site (with several players receiving cards on each "hand"), or that 33,662 pairs of cards have been dealt to players (over a presumably smaller number of hands that have been dealt)?

 

[kmccormick100]

Well this is disconcerting. I’ve logged a fair number of hours on mavens over the last 6 months believing it to be neutral randomly generated hands, as there wouldn’t seem to be any incentive for skewing the cards. My only thought would be that somehow they thought by generating more big starting hands players would have more fun/action in their games? I have observed that it feels like I get a lot of pocket pairs, but chalked it up to the sheer number of hands online vs live, which I’d never really played much before this year (since Black Friday). Anecdotally I did pickup aces and kings twice each in like an hour the other night and think it was unusual.

 

[Senzrock]

Well this is disconcerting. I’ve logged a fair number of hours on mavens over the last 6 months believing it to be neutral randomly generated hands, as there wouldn’t seem to be any incentive for skewing the cards. My only thought would be that somehow they thought by generating more big starting hands players would have more fun/action in their games? I have observed that it feels like I get a lot of pocket pairs, but chalked it up to the sheer number of hands online vs live, which I’d never really played much before this year (since Black Friday). Anecdotally I did pickup aces and kings twice each in like an hour the other night and think it was unusual.

I think once @BearMetal responds to the questions above, we will get a better sense of things. Right now I wouldn't be too concerned as it seems like the %'s weren't calculated correctly.

 

[Jimulacrum]

And this isn't a case of pocket pairs, especially AA being strongly skewed because they *are shown down* vs many other types of hands?

This is a really good point, as are the few comments I've seen pop up under this one.

AFAIK, Mavens doesn't offer the ability for the host to see other players' cards (during or after the fact) unless they went to showdown or were voluntarily shown by the player. If this data is being drawn only from hands that were shown, it would make sense for it to be skewed in favor of pockets pairs (and big aces, Broadway combos, etc.). And if Mavens does in fact allow the host to see all hands, that's kinda messed up and would make me reconsider using it.

As to the royals, I've been dealt 11 royals myself, about half in Hold'em and the rest in Omaha variants and circus games. I've played somewhere around 5,000 hours of poker, lifetime. I know people who've played about as much as I have, if not more, who haven't been dealt any royals, or some low number like one or two. The topic of straight flushes doesn't come up as often, but I believe I've been dealt more of those than average too (not including royals). Variability in data can be crazy sometimes.

 

[JustinInMN]

It's hard to break down this information since this only represents the hands that were played/shown;

So, I did a much simpler test: I took a look at how often pocket pairs come up.

The definition of "how often pocket pairs come up" is VERY IMPORTANT here.

Do you mean at showdown? Because I would expect pocket pairs to be a much higher percentage of hands actually shown down on average. Without getting too deep in the strategy weeds, there are some situations to fold smaller and medium pairs, but in a majority of situations a majority of pairs are playable. Where as in almost all situations it is right to fold at least 50-60% of unpaired hands preflop, and some situations even more than that. (Sometimes even 100% of unpaired hands should be folded.)

If you could run an analysis of all starting hands that would be a better indicator of how close you are in frequency to a probability baseline. If you dealt 200 10-handed hands over a night, that's 2000 starting hands. Of that AA should come up one in 221 or about 9 times, for example. (And I don't think anyone is going to cry rigged if one night you get 6 and another you get 12.) But if we are only looking at showdown hand, well that universe is much small, depending on how sticky your group is at the river, and average number of hands shown down per deal could be less than one to maybe around two. That cuts the haystack we are researching down significantly, like by 90 percent, and the hands that remain would tend to be much richer in pocket pairs for the strategic reasons above.

Data is kind of my gig. We need to adequately define the haystack and the needle before we can decide if there is an issue. I think chances are the original analysis is being applied to a different haystack incorrectly.

 

[BearMetal]

And this isn't a case of pocket pairs, especially AA being strongly skewed because they *are shown down* vs many other types of hands?

No; this data is raw data. PM keeps track of hands not shown as well. This is _DEALT_ pocket pairs.

 

[BearMetal]

This math doesn't add up to me. A specific pocket pair should appear once in every 221 hands per player. More players at the table = more pocket pairs. You should either look at a single players dealt hand history or figure out how many players are at the table at each point, which seems way more difficult to do with just some greps.

This is why I did the math with my average number of players (I get between 11 and 15, but over 80% of the time it's 13 to 14 players). So, with 14 players at the table and a pocket pair once every 3 hands, that means that each player has about a 1 out of 42 chance of getting it, which is still 5x as high.

A specific pocket pair (like AA) should only appear every 221 hands, but any pocket pair should appear once every 17 hands (3/51).

Also, when you say you've seen 33,662 hands, do you mean that 33,662 hands have been dealt on your site (with several players receiving cards on each "hand"), or that 33,662 pairs of cards have been dealt to players (over a presumably smaller number of hands that have been dealt)?

A "Hand" means that it was dealt to the table. The multiplication of the 1-of-3 summary by the total players should cover this. On average, I have 14 players, so we can simplify this by multiplying 33,662 by 14 and then dividing by the total pocket pairs, which gives us roughly 1-out-of-46; still 5x more. If it was 3x, OK, but 5 seems out of line.

The definition of "how often pocket pairs come up" is VERY IMPORTANT here.

Do you mean at showdown? Because I would expect pocket pairs to be a much higher percentage of hands actually shown down on average. Without getting too deep in the strategy weeds, there are some situations to fold smaller and medium pairs, but in a majority of situations a majority of pairs are playable. Where as in almost all situations it is right to fold at least 50-60% of unpaired hands preflop, and some situations even more than that. (Sometimes even 100% of unpaired hands should be folded.)

If you could run an analysis of all starting hands that would be a better indicator of how close you are in frequency to a probability baseline. If you dealt 200 10-handed hands over a night, that's 2000 starting hands. Of that AA should come up one in 221 or about 9 times, for example. But if we are only looking at showdown hand, well that universe is much small, depending on how sticky your group is at the river, and average number of hands shown down per deal could be less than one to maybe around two. That cuts the haystack we are researching down significantly, like by 90 percent, and the hands that remain would tend to be much richer in pocket pairs for the strategic reasons above.

Data is kind of my gig. We need to adequately define the haystack and the needle before we can decide if there is an issue. I think chances are the original analysis is being applied to a different haystack incorrectly.

I mean it is dealt to the player. PM keeps track of hands dealt even if not shown.

 

[kmccormick100]

@RainmanTrail also knows a thing or two about data and statistics and poker sample sizes. Any thoughts Trav?

 

[BearMetal]

So, let's break this down to a single player who has played in every tournament.

• He was dealt 9,385 hands
• Here are his pocket pairs (AA to 22):
  • 106
    88
    113
    106
    118
    116
    80
    112
    121
    112
    111
    113
    129

Every 88 hands, he receives AA. That's still 3x as likely to get it, no?

 

[Finest]

AFAIK, Mavens doesn't offer the ability for the host to see other players' cards (during or after the fact) unless they went to showdown or were voluntarily shown by the player.

During a hand, only a player holding cards can see those cards. After a hand has been completed, a Mavens site admin can see all the cards that each player held, regardless of whether the hand went to showdown.

 

[upNdown]

So, let's break this down to a single player who has played in every tournament.

• He was dealt 9,385 hands
• Here are his pocket pairs (AA to 22):
  • 106
    88
    113
    106
    118
    116
    80
    112
    121
    112
    111
    113
    129

Every 88 hands, he receives AA. That's still 3x as likely to get it, no?

This is disturbing.

During a hand, only a player holding cards can see those cards. After a hand has been completed, a Mavens site admin can see all the cards that each player held, regardless of whether the hand went to showdown.

This, more so.

 

[BearMetal]

During a hand, only a player holding cards can see those cards. After a hand has been completed, a Mavens site admin can see all the cards that each player held, regardless of whether the hand went to showdown.

And that's why I always say "You can only trust as site as much as you trust its operator". My players trust that I frankly don't care about past hands (aside from statistics) and I don't go snooping. Frankly my time is too valuable; I've got snappers to clean!

 

[JustinInMN]

So, let's break this down to a single player who has played in every tournament.

• He was dealt 9,385 hands
• Here are his pocket pairs (AA to 22):
  • 106
    88
    113
    106
    118
    116
    80
    112
    121
    112
    111
    113
    129

Every 88 hands, he receives AA. That's still 3x as likely to get it, no?

Good, I appreciate the simplification here. Makes my next question easy.

Does 9385 represent every single hand he is dealt, or is there anything this search wouldn't catch? (Such as folds before the river?)

 

[Senzrock]

This is disturbing.

This, more so.

The 1st (if accurate, and I think ~10k hands is still relatively small sample size for online play, even if correctly calculated), is way more concerning than 2nd. That site admins have access to card information after completion of hands should be known by players (there have been threads on it here in the past I know). As mentioned, there has to be some level of trust between site operators + player pool.

 

[CraigT78]

During a hand, only a player holding cards can see those cards. After a hand has been completed, a Mavens site admin can see all the cards that each player held, regardless of whether the hand went to showdown.

I think this needs to be reinforced.

No one can see hands that are in play. They aren't logged until after the hand completes.

Backend Server access is needed to view logs. They are not available to a player or an admin who does not have backend server access.

 

[Silver_Fiend]

Does the website take a rake? If they do, then theres the reason to skew cards. I quit playing on a free-play site because of the same thing. I saw more quads, straight flushes and royals in a few months than in 15 years of playing live cards. The reason is the rake. Yes, even freeplay sites take a rake to eliminate chips, so players are more apt to purchase the play chips. Bigger pot = bigger rake = more chips sold.

 

[Senzrock]

Does the website take a rake? If they do, then theres the reason to skew cards. I quit playing on a free-play site because of the same thing. I saw more quads, straight flushes and royals in a few months than in 15 years of playing live cards. The reason is the rake. Yes, even freeplay sites take a rake to eliminate chips, so players are more apt to purchase the play chips. Bigger pot = bigger rake = more chips sold.

There is a rake option (some people use the software for income generating "poker rooms") but standard is without rake. Few of us play in a rake free room, which in turn also means our player pool is much smaller because of built in level of trust etc.)

 

[madforpancakes]

So, let's break this down to a single player who has played in every tournament.

• He was dealt 9,385 hands
• Here are his pocket pairs (AA to 22):
  • 106
    88
    113
    106
    118
    116
    80
    112
    121
    112
    111
    113
    129

Every 88 hands, he receives AA. That's still 3x as likely to get it, no?

This is more convincing, but this is still a fairly small sample size right? Ken has this utility available on his site that uses the same shuffling algorithm and RNG that the main mavens program uses: https://www.briggsoft.com/docs/pmavens/Utilities.htm#shuffle

 

[upNdown]

This is more convincing, but this is still a fairly small sample size right?

In 9385 hands the player "should" get each pocket pair 42 times. I'm not persuaded by a sample size argument, when he's getting EVERY pocket pair 2-3 times more than expected.

 

[shorticus]

I think this needs to be reinforced.

No one can see hands that are in play. They aren't logged until after the hand completes.

Backend Server access is needed to view logs. They are not available to a player or an admin who does not have backend server access.

So essentially only the server admin can get access to this information?

 

[Senzrock]

So essentially only the server admin can get access to this information?

Correct (edit: + only after hands are completed, never mid-hand).

 

[AWenger]

So, I did a much simpler test: I took a look at how often pocket pairs come up. Turns out, AA came up nearly 5x as often as it should. A pocket pair should appear once every 221 hands. Out of 33,662 hands, that means it should appear about 152x. I ran this query for all pocket pairs, and it turns out they are all 5x as often:

over 33,662 hands which is roughly 1 out of every 3. I typically have about 14 players on average, so if you say that every 3 hands, 1 out of my 14 players gets a pocket pair,

A couple questions I didn't see explained or specified anywhere in the posts (but maybe I missed it), so I'll ask them:

1) Are these hands for Hold'em only? (i.e. no Omaha 4 or 5-card games mixed into the total hands?)

2) Is 33,662 the number of hands dealt to the table (table-hands) or to the players (player-hands)? For example, if there are 6 players on a table, 1 hand dealt would be 1 table-hand, and 6 player-hands. Depending on the answer, shouldn't the number of hands dealt, [33,662] be multiplied by the number of average players at table, to get the total number of player-hands dealt?

3) When you say you "have about 14 players on average" what exactly does this mean? How many players are playing at one table there the cards are dealt? Are you running 2-tables with 7 players each? Do the tables ever run short-handed, say with 3, 4, or 5 players?

 

[BearMetal]

Does 9385 represent every single hand he is dealt, or is there anything this search wouldn't catch? (Such as folds before the river?)

It represents ALL the hands he was dealt. The search catches them all; folds included.

The 1st (if accurate, and I think ~10k hands is still relatively small sample size for online play, even if correctly calculated), is way more concerning than 2nd. That site admins have access to card information after completion of hands should be known by players (there have been threads on it here in the past I know). As mentioned, there has to be some level of trust between site operators + player pool.

Yes, I know that my site stats aren't from a huge pool; but they are consistent across players which is why I was concerned. All my players know that the hands are captured after the tournament ends. There's actually another troubling issue where sometimes it's just "too good to be true"; two aggressive players are dealt AA and KK in a huge pots and the flop is AKAK7 ... I mean, come on. But I am NOT complaining about that here. I'm still attributing this to just the shear numbers of hands played; I don't think the software is manufacturing hands to make games more exciting.

No one can see hands that are in play. They aren't logged until after the hand completes.

Mine aren't logged until after the tournament ends.

Does the website take a rake? If they do, then theres the reason to skew cards.

My site takes no rake; it's just for close family/friends. The license fee was paid once, so there's no incentive to make the software more exciting to gather more money from players.

This is more convincing, but this is still a fairly small sample size right? Ken has this utility available on his site that uses the same shuffling algorithm and RNG that the main mavens program uses: https://www.briggsoft.com/docs/pmavens/Utilities.htm#shuffle

Yep; I made reference to this. I looked it over and it seemed legit. When I ran it, it produced fewer pocket pairs over the same # hands. We can't 100% be certain this is the algorithm in use unless someone decompiled the software. But I have no reason to believe that PM would advertise one algorithm and use another. I am not accusing PM or Briggs of any wrong-doings; I'm just sharing my raw stats in hopes I can find some reason...

In 9385 hands the player "should" get each pocket pair 42 times. I'm not persuaded by a sample size argument, when he's getting EVERY pocket pair 2-3 times more than expected.

That's my point; all of the pocket pairs are too much for this 1 player.

1) Are these hands for Hold'em only? (i.e. no Omaha 4 or 5-card games mixed into the total hands?)

2) Is 33,662 the number of hands dealt to the table (table-hands) or to the players (player-hands)? For example, if there are 6 players on a table, 1 hand dealt would be 1 table-hand, and 6 player-hands. Depending on the answer, shouldn't the number of hands dealt, [33,662] be multiplied by the number of average players at table, to get the total number of player-hands dealt?

3) When you say you "have about 14 players on average" what exactly does this mean? How many players are playing at one table there the cards are dealt? Are you running 2-tables with 7 players each? Do the tables ever run short-handed, say with 3, 4, or 5 players?

1. These are hands from 2 recurring tournaments which are both NLHE
2. That's the number of table hands; 1 table hand = 14 player hands. This is why I broke it down by multiplying by the average number of players I have. For a more precise sample set, I isolated a single player across all tournaments (yes he plays in every one) and you can see the numbers are still inline with being too often.
3. Yes, you bring up a good point here. My tournaments usually start with 2 tables, 7 players, but quickly wind down to a single table. There's going to be some noise caused by this, so that's why I ran the single-player stats above.

 

[Jimulacrum]

So, let's break this down to a single player who has played in every tournament.

• He was dealt 9,385 hands
• Here are his pocket pairs (AA to 22):
  • 106
    88
    113
    106
    118
    116
    80
    112
    121
    112
    111
    113
    129

Every 88 hands, he receives AA. That's still 3x as likely to get it, no?

It's about 2.5x, not quite 3x. But this is the problem with (relatively) small samples. Does this player's openers represent an average result compared to other players' data? Above average? Below average? Just about every data set is going to have outliers, especially if we're talking about something that's randomized independently every iteration.

In theory, each player should get each pocket pair about an equal number of times as each other pocket pair, which clearly isn't the case here. At the extremes, we have 80 and 129, representing being dealt 22 more than 50% more often than 88. It doesn't mean the site favors 22 or disfavors 88; it's just random variance, like a coin landing on heads a bunch of times in a row. It's a little odd for one player to get an above-average result for all of his pocket pairs, but not any more odd than a player getting a below-average result for all of his pocket pairs, which of course can happen too.

Also, if your hypothesis is that the site is trying to stir up action with unequal hand distributions, shouldn't this pocket-pair data be more top-heavy? Little pairs don't stir up big action; big pairs do. And in this same stroke, I'd like to point out something you addressed right upfront, which is that Briggs has no motive for his software to produce skewed results, whereas he does have motive for it to be fairly randomized. Why should he care if his site stirs up a little more action than others, especially if using broken algorithms to achieve it could drive people away from buying his software?

Out of 33,662 hands, that means it should appear about 152x. I ran this query for all pocket pairs, and it turns out they are all 5x as often:

I find it odd that your data on one player includes 9,000+ hands, while your total data pool only covers about 34,000 hands. It makes me feel like you're looking at all of the times all the different pairs have come up across all the players at the table, but still only comparing it to 33,662. If you have 33,662 hands, but 9 players were at the table for each hand, you're really looking at 302,958 openers. Most likely, your data represents somewhat fewer hands than this, since I doubt you're 9-handed all the time, but still way more than 33,662 openers.

Maybe we can test this over a large number of dummy trials. If I understand Mavens (I have a server too but haven't hosted anything yet), you can set up a table that will essentially auto-deal a ton of hands in a relatively short period, without having to involve any real players. Just make two dummy accounts, and make a cash table that's cap-limit Hold'em, with an ante equal to the cap, and duplicate IP addresses allowed. Give both dummy accounts a bazillion chips in their balance and let them buy in for the whole thing. Then just sit them down and let it roll.

In fact, I think I'll take part of my lunch break to see if I can set this up on my server. The part i don't know how to do is run queries on past hands, though, so I may swing back here and ask for help with that.