In addition, I now manually make adjustments to scores when players report a problem. This is very time consuming and simply not sustainable or scalable (as we add more players).
I therefore plan to implement a forgiveness policy starting with Q3 2010. We will "forgive" a certain number of missed days or low scores per month/quarter. Once this is in place, we will also no longer make adjustments to scores, such as when your quiz submission is interrupted and you have to go back and "take the quiz" a second time.
I share with you below my ideas on how to implement/apply this policy. I would love to hear what you think. It may take a little while to sort it all out, but of this you can be certain:
The forgiveness policy will be applied at the end of Q3 (and every quarter thereafter). The determination of participation in the championship playoff will be made after I apply the forgiveness algorithm. So if you miss a few days of quizzes, do not panic!
- Must be applied equally to everyone. That is, if I do not miss any quizzes, I should not be penalized, relatively speaking. Even I should see a benefit from the forgiveness policy.
- Apply forgiveness to 10 days of quizzes in a quarter(out of about 60 days).
- For each missed quiz (up to 10), give that person a score equal to average of all scores for that day. This way, you do not take the "full hit" of a 0 score, but you also will not "jump ahead" of those who rank lots higher than you (and remember: this only happens for a small number of days out of the total).
- If you missed less than 10 quizzes in a quarter (let's call the balance N), then I will apply a different algorithm to the N lowest ranked days in that quarter. Suppose, for example, you usually rank in the top 100, but on 15 days during the quarter, your ranking was around 200. You also missed 3 quizzes. Then I would "forgive" 7 of the lowest ~200 ranked days.
- Forgiveness for low ranking: change your score for those days to match the score of the ranking on that day that equals your average ranking throughout the quarter. In other words, if you average ranking is 125 then on a day in which you were ranked 250, I find the score of the person ranked 125, give you that score, and then re-rank.
Thanks in advance for your thoughts.