An update regarding very fast answers

I received this email today from a devoted player:

"I continue to wonder how fast some people manage to submit. Like "Neha Jain", only three seconds to check three questions before hitting the submit button. How's that possible even when realizing that of course no question text could possibly be read. I've used to period-type "Week" report to get to the conclusion above. The same holds for the next conclusion: Well, Neha just answered one question this week. But what about Alen Oblak, nr 23 Overall Rank. Just 37 seconds to answer all four questions (almost without error) this week? What can we learn from Alen ? I was getting curious because every day on 'Past Quizzes' the maximum score of the day is mentioned, which is always higher than reasonably possible for anyone not just hitting some buttons very quick at random."

At first, I hesitated to include the names this player mentions in his email, but that's the beauty of the way I am running the PL/SQL Challenge (not to sound boastful :-) ). We are open about this data so everyone can see - and wonder the same things.

When we upgraded to 1.4, I published a change in the rules that sent out a rather severe warning to people who submit very fast answers. I plan within the next week or so to apply these rules (and a modification of them, which will be published in the Rules section this weekend with the ugprade to 1.5).

Here's the bottom line: you are about to see significant changes in the rankings of players who seem to be able to consistently play the quiz dramatically faster than even I (a PL/SQL expert and author of the quizzes) am able to simply read my own questions.

As I have mentioned in earlier blog postings, I suppose it is theoretically possible for someone who is an incredible speed reader; amazing PL/SQL expert; and very lucky individual to answer much faster than I can manage, but that behavior cannot be distinguished from those who cheat. So all I can do, to preserve the credibility of the PL/SQL Challenge, is discourage players from following that pattern.

So, my friends, just a little bit more patience and then those rankings will start to make a bit more sense.

In fact, again for the sake of transparency, you will find below my plans for adjusting timings, scores and rankings for VFAs (very fast answerers). Feel free to let me know what you think.

Planned Algorithm

1. I record the time it takes me to quickly answer each quiz. I consider this a reasonable minimum (RM), since it is actually an aggressive score, as I know PL/SQL pretty well and I also wrote the quiz (making it hard to avoid rapidly identifying the correct and incorrect answers).

2. Every two weeks (10 quizzes), I identify all players whose total time is less than mine (call them TFP, too fast players). I choose every two weeks because (a) I don't want to compare and adjust/punish on a daily basis. I need to allow that some people will know a feature better than me and answer more quickly and (b) 10 quizzes should be enough to reveal a pattern. Over two weeks, I would expect my RM to be around 300 seconds or 30 seconds per quiz on average.

3. For each TFP, if the delta is within 10 seconds, I do nothing; that's close enough and nothing to worry about.

4. If TFP is more than 10 secs faster, then for every 10 seconds faster (or portion thereof), I add 60 seconds to the time for each quiz answered faster than I did on each day.

So if my RM is 300 seconds and a person answers in 280, and 5 of the scores are less than mine, then their overall time with increase by 5 minutes or so (5 x 1 x 60 = 300). This should be enough to cause a dramatic decrease in their ranking and be a substantial incentive to SLOOOOW down or stop cheating. If they only had a couple of too fast answers and the others are reasonable, then the impact on their is lessened.

Your thoughts?

SF