Alright, so prepare yourself, there's gonna be a lot of math in this post. I ran this competition's score reports through some different totalling methods, trying to find the fairest method that both reduces the team penalty for teammates not reporting, while still maintaining some decent incentive for actually reporting your score and remaining simple enough to understand so there's no incredible score surprises.
You can see the results of my methods in this spreadsheet: https://docs.google.com/spreadsheets...it?usp=sharing
First off, I'd say that if every member of the comp had full participation and was in the proper tier for their skill, there shouldn't be more than a 5-7% score difference between the first place and last place teams. So that's the basic guideline for deciding if a scoring method was feasible.
Current Method - Sum
Nightly Total Determined By: Adding all the members scores together each night
Formula for Cell B15: =SUM(B3:B14)
Pros: Simple to understand, and just feels natural
Cons: Lack of participation can cause drastic deficits in team scores.
Example of Method Flaw: In this competition, the Vampires beat the other teams by 400/800 points, despite the fact that they only had one member in the Top 5 scorers for the beginner or intermediate tiers, and not having the top scorer in the expert tier. This enormous deficit was mainly caused by missing members on the other teams, including 5 of the 12 members of the Werewolves not posting anything after the first week.
Outcome (without Tier Bonuses):
Vampires - 3021
Skeletons - 2600.5
Werewolves - 2220
% Difference of First and Last Scores: 26.5%
Average Method
Nightly Total Determined By: Averaging (arithmetic mean) the reported scores for each team.
Formula for Cell B15: =AVERAGEIF(B3:B14,"<>0")*12
Note: I multiplied by 12 to make the totals more comparable with the other methods. Since it is a constant number across all teams, this doesn't affect the ultimate outcome.
Pros: No penalty for non-participating team members. Equals the playing field for teams of different sizes.
Cons: Could create lack of incentive to post low scores.
Example of Method Flaw: Say I scored 0 points on a night, but the average from the other 11 members on my team is 15. Adding my 0 points would drop the team average to a 13.75. Thus, it is more advantageous for me to not post my score to give the team an extra 1.25 points (a full 15 point difference if using the *12 method I used in the spreadsheet).
Outcome (without Tier Bonuses):
Werewolves - 4603.95
Skeletons - 4044.09
Vampires - 3903.85
% Difference of First and Last Scores: 15.2%
50% Average Method
Nightly Total Determined By: Adding the scores for each member per night, replacing non-reported nightly scores with a value at 50% of the team average.
Formula for Cell B15: =SUM(B3:B14)+COUNTIF(B3:B14,"")*(AVERAGEIF(B3:B14, "<>0")*0.5)
Pros: Improves slightly on the previous method's lack of incentive con.
Cons: Still may be tactically advantageous to not report scores. Also will penalize more for unreported expert scores as opposed to intermediate or beginner scores.
Example of Method Flaw: Say I scored 0 points on a night, but the average from the other 11 members on my team is 15. Instead of adding my 0 points and keeping the team score at 165, it is more advantageous for me to not post my score, causing my score to be replaced with half the average and giving my team an extra 7.5 points for the night.
On the opposite side, say an expert doesn't post their score. Typically experts average around 50-60 points per night, but if they didn't report, they would be replaced with a 7.5, causing about a 50-point hit to their team.
Outcome (without Tier Bonuses):
Vampires - 3462.42
Werewolves - 3411.97
Skeletons - 3153.79
% Difference of First and Last Scores: 8.9%
50% Team Tier Average Method
Nightly Total Determined By: Adding the scores for each member per night, replacing non-reported nightly scores with a value at 50% of the average for the rest of their teammates in that tier. In other words, a non-reported score for a beginner would be replaced with 50% of the average of the other beginners on their team.
Formula for Cell B15: =SUM(B3:B14)+(COUNTIF(B3:B4,"")*(AVERAGEIF(B3:B4," <>0")*0.5))+(COUNTIF(B5:B9,"")*(AVERAGEIF(B5:B9,"< >0")*0.5))+(COUNTIF(B10:B14,"")*(AVERAGEIF(B10:B14 ,"<>0")*0.5))
Pros: Same as the previous, but also removes con of penalizing higher tiers for the naturally lower scores of lower tiers.
Cons: Can cause a point deficit if all members of a single tier don't report.
Example of Method Flaw: On night 18, both the experts on the Skeletons had a blank score, initially causing a #DIV/0 error on the spreadsheet and (more importantly) causing the them to have a 0 average for experts, significantly impacting their score.
(After reviewing the spreadsheet, RelaxAndDream actually scored 0 points that night, but because of how Excel/Google Sheets works, that caused the score to appear blank and therefore not be included in the average calculation. To remedy this, I gave RelaxAndDream a trivial score for the night, 0.00001, which allowed it to calculate the average but have no impact on the scores, which were rounded to 2 decimal places. If implementing this method, this will have to be used in the future for all "reported but zero" scores to differentiate from "non-reported" scores.)
Outcome (without Tier Bonuses):
Vampires - 3286.54
Skeletons - 3261.75
Werewolves - 3114.46
% Difference of First and Last Scores: 5.2%
For the record, of the options listed, I think this one is the best as far as weighing pros and cons goes.
50% Overall Tier Average Method
Nightly Total Determined By: Adding the scores for each member per night, replacing non-reported nightly scores with a value at 50% of the average for the rest of competition members in that tier. In other words, a non-reported score for a beginner would be replaced with 50% of the average of the other beginners in the competition, including those on other teams.
Formula for Cell B15: =SUM(B3:B14)+(COUNTIF(B3:B4,"")*(SUM(B$3:B$4,B$18: B$19,B$32:B$33)/COUNT(B$3:B$4,B$18:B$19,B$32:B$33))*0.5)+(COUNTIF( B5:B9,"")*(SUM(B$5:B$9,B$20:B$23,B$34:B$38)/COUNT(B$5:B$9,B$20:B$23,B$34:B$38))*0.5)+(COUNTIF( B10:B14,"")*(SUM(B$10:B$14,B$24:B$28,B$39:B$43)/COUNT(B$10:B$14,B$24:B$28,B$39:B$43))*0.5)
Pros: Same as previous, but also removes con of one team missing all the members in a tier.
Cons: Incredibly convoluted scoring method. Also, having an exceptional score can inadvertently benefit the other teams as well.
Example of Method Flaw: Despite being in the beginner tier, Nazrax outscored all but 4 of the intermediate members. On night 11, Nazrax scored 70 points, bringing the average beginner score that night from about 7.29 to 15.13. Thus, unreported beginner scores would be replaced with a 7.56. So though Nazrax scored 70 points for the Vampires, he also inadvertently scored about 15 points for the Skeletons and about 23 points for the Werewolves due to their missing people.
Outcome (without Tier Bonuses):
Vampires - 3335.61
Skeletons - 3177.43
Werewolves - 3118.79
% Difference of First and Last Scores: 6.5%
So that's my comparison. If you guys have any other methods you want me to try plugging into Excel, let me know. Another thing that I didn't try was only applying these scoring methods to "inactive members", instead of "inactive nights". So some of the cons involving advantages for not reporting a particular night could be circumvented by only applying the compensated scoring for members who have been AWOL for longer than a week, and not to just single instances of missing scores. However, I greatly imagine that implementing that in the live competition scoresheet would be incredibly difficult, if not mostly impossible. Still, if it makes the competition fairer for everyone, it's a challenge I'm willing to tackle.
TL;DR - Current method sucks, here's me trying some alternatives.
|
|
Bookmarks