Moscow, when it comes to rating players, there are a lot of ways to do it. I think the first thing to do is try to figure out what you want to measure. What are your ratings supposed to determine?
Poker lends itself to so many different ways to run a tournament, let me use football for just a moment to explain. Looking at high school, colleges, and pros, high school rules vary by state. There are some real differences that can be outcome determinative. Colleges play by one set of rules, but each conference does things a little differently. The pros have everyone playing by one set of rules interpreted the same way. Most high schools play with 12 minute quarters; colleges and pros with 15 minute quarters, but measured differently. What they all have in common is this – the team with the most points at the end of the game wins. And for the most part, the way you score points is similar.
In poker, there will be huge differences between winners depending on game(s) played; how deep the stacks are; how the blinds go up; how long the blind rounds are; whether there are re-buys, add-ons, and if so how many; whether you play with one or two decks of cards; rules; etc. Even the day of the week might alter things. If you have players with normal day jobs, you might have less time during the week than on weekends. All of those variables will affect your results.
I personally like to see different formats, but I’m even more interested in the why behind a format. So I’ll explain what I was trying to measure and how I decided to try to measure it.
I’m limited to doing it once a month so I have a monthly game. I wanted to measure the best overall performers over the course of a year. It seemed to me that to really measure that, we needed a consistent format for our monthly game -- same game, stacks, blinds, etc. Otherwise I’m measuring something different every time. I think if we played weekly, we could vary more and still generate valid results. I think weekly would be even better as a true measure since I think determining the best players takes a season more like baseball (162 games) than high school or college football (10-12 games). 10-12 games is just very short.
I tried to figure out what things in poker are measurable. By doing only freeze-out tournaments, I eliminated buy-in amounts as a measure. That got me down to 6. One, money won, to me long term is the most valid criteria, but for a league, I think it’s a lousy thing to measure. The reason is some players come to have fun and play for social reasons. They don’t mind the competition and ratings, but if they are constantly reminded that they are net donators, they are less likely to keep coming. It’s not good for their ego, and it’s not good for your game. So I don’t use that criterion.
Our players are evaluated on 5 criteria. From easiest to accumulate to hardest to accumulate those are [1] points, [2] how many other players they knockout of a tournament, [3] final table appearances, [4] in the point finishes, and [5] tournament wins. We measure players on overall performance for the year, and then we also divide each category by per game performance. Each of those counts equally (10% of standings), but they aren’t really equal.
Suppose we have 30 players. There will be 327 points for the tournament; 29 KO’s (assuming no chop), 10 final table appearances, 7 in the point finishes, and 1 tournament win. Points will range from 102 down to 3 per player. You could argue that points and in the point finishes are the same, though I do think they measure something slightly different. Points measures how high you finished while in the points measures how often you finished in the top 7. In the points is similar to final table, but I think someone who made the final table deserves something more than those who didn’t. Those are the reasons I measure things the way I do.
We have up to 30 players (3 tables). Our scoring system is based on one from Bluff Magazine. Points are made up of two factors -- participation points multiplied by their finish points. Participation points = # of entrants/10. If we have 25 players, every player gets 2.5 points, and the in the money places are multiplied by that number. At 25, we pay 6 places. Here’s how the finish points work – 34, 21, 13, 8, 5, 3, 2 for the 7 in the money places. All other places give 1 for finish.
Players
Points to the top 7 -- 34, 21, 13, 8, 5, 3, 2 (for last in the points position)
28-30 – pays 7 places
21-27 – pays 6 places
15-20 – pays 5 places
10-14 – pays 4 places
6- 9 – pays 3 places
A 30 player tournament would work like this:
1st – 102 pts
2nd – 63 pts
3rd – 39 pts
4th – 24 pts
5th – 15 pts
6th – 9 pts
7th – 6 pts
All others – 3.0 pts.
Each player’s score every category is then compared to all other players. They get a score for a category like a baseball batting average (.387 for example) that represents the percentage of points they got compared to every other player.
While I measure 10 and each counts 10%, you can look at the numbers and realize that tournament wins will boost you more than any other thing for two reasons. It gets you the most points for the tournament, and there are only going to be 12 wins in the year. Thus, a player who gets 2 wins will get .167 in that category, while someone with no wins will get .000. If that player entered only 10 games, his score for per game performance will be .200.
For final table, we have 1 tournament a year where we only take 9, but all others, up to 10. That means at most there would be 119 final table appearances for the year. Since it’s possible for two or more to be KO’s while on the bubble, it might be fewer. There will be 84 in the point finishes. You could roughly estimate that a final table appearance is worth 84/119 of an in the points finish. So even though we weigh them equally, they aren’t equal.
The system rewards players according to how many participate whether they finish at the final table, in the points, or win.
I can’t say that the way we’ve done is perfectly balances the varying factors, but so far, no one has convinced me there is a better weighting system.
Our tournaments are deep stack – 25,000 to start, SB/BB 25/50 (start with 500 BB), 20 min blind levels, designed to end with 30 players at the 4:00-4:20 blind level (level 13).
All of our players pay the same amount for each tournament. Our end of the year game is a Main Event type event – bigger buy-in, longer planned tournament, and bigger payouts, but all participants start equal. It doesn’t count in our standings except to break a tie.
I don't think our way is better than all other ways. But I do see some people who put together formats that I do not think measure what they say they are trying to measure. For example, in multi-table events, I don't think it makes much sense to try to distinguish between the first two players out when others might show up later and enter. That rewards those who show up late. We instead reward quality performance, not better degrees of poor performance. I think ours does a good, but not necessarily great, job of measuring what I'm trying to measure.