Placing Rate

Why a smart game deserves smarter numbers.

SmilesLiesGunfire

July 25, 2024

The goals for this site are:

  1. Improve analysis of Kill Team through objective data.
  2. Do so in a way that is accessible and digestible to the average player.

With these ends in mind, I find it appealing to compare Kill Team statistics to professional sport statistics. After all, both subjects are focused on analyzing competitiveness to a lay audience.

There are some obvious differences (a few hundred billion 💰 you might say). But a relevant difference is the sheer volume of key statistics utilized in the world of sports.

By key statistic, I mean a critical statistic that summarizes performance in an objective but approachable way1. For example, in Baseball (the most statistic obsessed sport of all time) there are three major offensive stats known as Batting Average, On-Base Percentage, and Slugging Percentage. These performance indicators are well known among baseball fans; no one needs a doctorate in math to understand them.

On the other hand, faction analysis within the Games Workshop bubble is almost exclusively concerned with Win Rate. The reason for this is obvious: Win Rate is both easy to conceptualize and genuinely valuable.

Essentiallly, it makes for a great key statistic.

There really hasn’t been any other measure that has managed to achieve that optimal combination of value and simplicity.

Still, is it not possible that a bunch of nerds (who play make-believe wars driven by mathematics) would be interested in more tools in the statistical toolbox?

The Loneliest Number

Maybe it’s worth asking, what is Win Rate missing?

After all, "Win Rate isn’t everything", it has some obvious weaknesses as a lone statistic. Two that come to my mind are:

  • There are Gatekeeper factions that put up high Win Rates but struggle to win events.
  • There are Sleeper factions that may have average Win Rates; But, in the hands of skilled players, perform exceptionally well at events.

Both these issues suggest there can be an inconsistency between a faction’s ability to win games and their ability to finish well at events.

Which should make sense. Competitive players aren’t just trying to win games, they’re trying to win events.

Perhaps what this bachelor needs is a companion that measures how well a faction finishes at events2?

Behold, my Stats

Here’s my attempt at this:

Placing Rate

And in English:

A faction’s Placing Rate is that faction’s total number of top_placings divided by their total number of picks.

Let’s unpack this.

First, what are picks? A faction pick is whenever a player takes that faction to an event (they "picked" that faction). If 3 players take Kommandos to one event, that’s 3 Kommando picks. If those same 3 players take Kommandos to another event; we now have 6 Kommando picks. To get a faction’s total number of picks, calculate the sum of all its picks for every event in the dataset.

Easy. Now, what are top_placings?

That’s when a player (i.e. a pick) finishes in a top placing at an event.

Okay
 but what makes a placing a top placing?

Well, that depends. Most people understand a top placing to be 1st, 2nd, or 3rd place (which is what we call "podiums"). Although intuitive, only measuring the top 3 placings for events is a very bad approach from a statistical standpoint. Kill Team GT events range from 16 to sometimes over 100 people. Obviously, it is much easier to get 3rd place at a 16-person event than an 100-person event.

Instead, a better alternative would be to calculate top placings by the percentage of all players attending that event. For example, a 15% top percentage would yield 1st through 3rd as top placings at a 20-person event. Additionally, that same 15% top percentage would yield 1st through 12th as top placings at an 80-person event3.

This is comparable to what poker events do when determining which players finish "in the money". Regardless of the tournament size, just take the top percentage of players (usually somewhere between 10% and 20%). Those players are the ones who walk away with something.

If it’s good enough for Vegas, it’s good enough for me.

Alright, now we know we should avoid static values. But if we’re using a top percentage, what percentage should we use?

I’m using 15% as an example because that’s my chosen value. Realistically, this decision is up for debate. If we use a lower percentage, such as 10%, we gain more validity but are sacrificing reliability. If we use a higher percentage, such as 20%, we gain reliability but are sacrificing validity4.

I prefer 15% as a healthy sweet spot, but there are other sensible values to use. What’s important is that we understand both the pros and cons that each percentage can bring to the results.

Finally, another great aspect of Placing Rate calculated from top percentage is it gives us a very clear expected_value. If I’m using a 15% top percentage for my top placings, then a perfectly average (i.e. balanced) faction is expected to produce a 15% placing rate.

Why? because if 15% of all placings are top placings, then 15% of a faction’s player base is expected to finish in those placings. Therefore, 15% becomes the average.

Let’s summarize all of this:

If we say Kommandos have a 20% Placing Rate. We mean 20% of all Kommando players finished in top placings at events. That also means Kommando’s Placing Rate would be about 5% above the average.

Pretty clear, right?

Advanced Placing Rate

Earlier I mentioned choosing a top percentage is a balance between reliability and validity.

Well, with the Advanced Placing Rate, I’m trying to have my cake and eat it too.

The basic idea is to calculate two Placing Rates together, but at different values. One Placing Rate is narrow but provides full value; meanwhile another Placing Rate is wide but provides less value. Here’s the formula:

Advanced Placing Rate

And in English:

A faction’s Advanced Placing Rate is that faction’s total number of narrow_top_placings (a top percentage of 12.5% always rounding down) plus total number of wide_top_placings (a top percentage of 17.5% always rounding up) divided by their total number of picks times 2.

Keep in mind, if a player places in the top 12.5%, they are also in the top 17.5% as well. This effectively causes individual 12.5% placings to have a value of 1 while individual 17.5% placings have a value of 0.5 (similar to how we do wins as 1 and ties as 0.5 in Win Rate).

A nice thing about this formula is that it averages out to a 15% Placing Rate (and it’s still a percentage, values will always be between 0 and 1). If you weren’t following anything I just said, you can still just approach this statistic with the same expectations as the basic Placing Rate.

Conclusion

The purpose of Placing Rate is not to replace Win Rate, but to supplement it.

Placing Rate is very good at highlighting factions I described earlier as Gatekeepers or Sleepers. It provides a fuller picture of a faction’s performance when combined with Win Rate.

However, Placing Rate is a very hungry statistic. It requires a lot of data to produce reliable results (in case you haven’t noticed, sample sizes are a concern in Kill Team event data).

The Advanced Placing Rate is an improvement that helps emphasize factions that are finishing in the highest placings, while still recognizing factions that are contenders. By squeezing a little more information out of our data, we can improve our statistics’ reliability without sacrificing validity. I’m currently using the Advanced Placing Rate in my quarterly stats because that’s where I focus most on reliability.

On that note, my next article will cover Kill Team sample sizes and how we can understand the reliability of our statistics.

Thanks for the read! đŸ§đŸ„ƒ

Footnotes

  1. Those familiar with Business Intelligence terminology would recognize this has a KPI. ↩

  2. That should get mom to shut up. ↩

  3. A top percentage is also agnostic to the type of system an event’s organizers use to rank their placings (such as a swiss or swiss with top-cut). Placing Rate doesn’t care. It only looks at the outcome. ↩

  4. Reliability verses validity is an extended topic in statistics. ↩