Congratulations first and foremost to our top-ranked players:
1st Place: Peter Auer of Germany
2nd Place: Mike Pargeter of United Kingdom
3rd Place: Frank Schmitt of Germany
This was Peter's first championship. An impressive rookie performance, Peter!
Next, congratulations to everyone who played in the championship. I hope you found it entertaining, challenging and educational. And for those who were not able to participate in the championship, you can take the quizzes next week through the Practice feature. We will also make the championship as a whole available as a Test, so you can take it just like these players did.
My thanks to all my fine reviewers, but most particularly Elic, whose impact on the quality of our daily quizzes and championships cannot be overstated.
Steven Feuerstein
Note: Below the table of results for this championship, you will find another list showing the championship history of each of these players.
| Rank | Name | Country | Total Time | % Correct | Total Score |
|---|---|---|---|---|---|
| 1 | Peter Auer (1) | Germany | 27 mins 25 secs | 92% | 2882 |
| 2 | Mike Pargeter (12) | United Kingdom | 22 mins 06 secs | 87% | 2788 |
| 3 | Frank Schmitt (8) | Germany | 30 mins 05 secs | 87% | 2723 |
| 4 | Karel Prech (3) | Czech Republic | 32 mins 25 secs | 87% | 2652 |
| 5 | Michal Cvan (11) | Slovakia | 33 mins 58 secs | 87% | 2551 |
| 6 | Zoltan Fulop (7) | Hungary | 29 mins 14 secs | 82% | 2485 |
| 7 | Janis Baiza (9) | Latvia | 22 mins 35 secs | 76% | 2443 |
| 8 | Jerry Bull (11) | United States | 29 mins 20 secs | 74% | 2443 |
| 9 | Ravshan Abbasov (1) | Uzbekistan | 33 mins 11 secs | 79% | 2441 |
| 10 | Anna Onishchuk (8) | Ireland | 16 mins 40 secs | 79% | 2432 |
| 11 | dmitrysk (1) | Russia | 32 mins 52 secs | 84% | 2398 |
| 12 | Manfred Kleander (1) | Austria | 34 mins 52 secs | 84% | 2393 |
| 13 | Vincent Malgrat (7) | French Republic | 32 mins 35 secs | 79% | 2388 |
| 14 | Lukasz Kubicki (1) | Poland | 22 mins 54 secs | 71% | 2362 |
| 15 | james su (8) | Canada | 21 mins 52 secs | 71% | 2348 |
| 16 | Chad Lee (10) | United States | 33 mins 00 secs | 84% | 2325 |
| 17 | Jeroen Rutte (7) | Netherlands | 34 mins 46 secs | 82% | 2320 |
| 18 | Frank Puechl (4) | Germany | 20 mins 50 secs | 74% | 2298 |
| 19 | Rytis Budreika (1) | Lithuania | 18 mins 20 secs | 68% | 2253 |
| 20 | mentzel.iudith (13) | Israel | 33 mins 13 secs | 84% | 2251 |
| 21 | João Barreto (4) | Portugal | 22 mins 17 secs | 74% | 2214 |
| 22 | Telmoc (2) | Portugal | 20 mins 30 secs | 66% | 2190 |
| 23 | Randy Gettman (12) | United States | 34 mins 52 secs | 71% | 2183 |
| 24 | Yuri Pedan (4) | Ukraine | 17 mins 06 secs | 71% | 2163 |
| 25 | Viacheslav Stepanov (12) | Russia | 31 mins 00 secs | 76% | 2145 |
| 26 | Krzysztof Helbin (3) | Poland | 26 mins 08 secs | 68% | 2112 |
| 27 | Anil Jha (3) | United States | 34 mins 56 secs | 76% | 2101 |
| 28 | Kevan Gelling (9) | Isle of Man | 30 mins 37 secs | 68% | 2078 |
| 29 | Siim Kask (13) | Estonia | 27 mins 48 secs | 63% | 2054 |
| 30 | Leszek Grudzień (2) | Poland | 30 mins 20 secs | 66% | 1998 |
| 31 | Yuan Tschang (8) | United States | 34 mins 24 secs | 55% | 1972 |
| 32 | Niels Hecker (14) | Germany | 35 mins 00 secs | 71% | 1960 |
| 33 | Oleksiy Varchyn (2) | Norway | 21 mins 32 secs | 61% | 1919 |
| 34 | _tiki_4_ (6) | Germany | 25 mins 06 secs | 58% | 1678 |
| 35 | Goran Stefanović (5) | Serbia | 24 mins 33 secs | 61% | 1544 |
| 36 | Rakesh Dadhich (5) | India | 26 mins 50 secs | 55% | 1443 |
| 37 | Stelios Vlasopoulos (10) | Belgium | 24 mins 13 secs | 50% | 1406 |
| 38 | Thierry Poels (7) | Belgium | 28 mins 05 secs | 58% | 1343 |
| 39 | Milibor Jovanovic (5) | Serbia | 32 mins 24 secs | 53% | 1272 |
| 40 | Dan Kiser (6) | United States | 33 mins 38 secs | 50% | 897 |
Championship Performance History
After each name, the quarter in which he or she played, and the ranking in that championship.| Name | History |
|---|---|
| Peter Auer | Q4 2013:1st |
| Mike Pargeter | Q4 2010:22nd, Q1 2011:15th, Q2 2011:8th, Q4 2011:5th, Q1 2012:5th, Q2 2012:17th, Q3 2012:5th, Q4 2012:20th, Q2 2013:12th, Q4 2013:2nd |
| Frank Schmitt | Q4 2011:22nd, Q2 2012:2nd, Q3 2012:21st, Q4 2012:3rd, Q1 2013:13th, Q2 2013:1st, Q3 2013:9th, Q4 2013:3rd |
| Karel Prech | Q3 2012:36th, Q4 2012:23rd, Q4 2013:4th |
| Michal Cvan | Q3 2010:22nd, Q4 2010:25th, Q3 2011:21st, Q1 2012:11th, Q3 2012:14th, Q4 2012:14th, Q1 2013:14th, Q2 2013:11th, Q3 2013:1st, Q4 2013:5th |
| Zoltan Fulop | Q1 2012:15th, Q2 2012:26th, Q3 2012:22nd, Q4 2012:17th, Q1 2013:22nd, Q3 2013:22nd, Q4 2013:6th |
| Janis Baiza | Q2 2010:3rd, Q4 2010:7th, Q3 2011:9th, Q4 2011:1st, Q3 2012:23rd, Q1 2013:4th, Q2 2013:4th, Q3 2013:3rd, Q4 2013:7th |
| Jerry Bull | Q2 2011:32nd, Q3 2011:8th, Q1 2012:12th, Q2 2012:11th, Q3 2012:13th, Q4 2012:15th, Q1 2013:9th, Q2 2013:15th, Q4 2013:8th |
| Ravshan Abbasov | Q4 2013:9th |
| Anna Onishchuk | Q1 2011:5th, Q2 2011:20th, Q3 2011:19th, Q4 2011:17th, Q1 2012:21st, Q2 2012:8th, Q4 2013:10th |
| dmitrysk | Q4 2013:11th |
| Manfred Kleander | Q4 2013:12th |
| Vincent Malgrat | Q4 2011:9th, Q1 2012:13th, Q2 2012:13th, Q4 2012:6th, Q1 2013:1st, Q2 2013:14th, Q4 2013:13th |
| Lukasz Kubicki | Q4 2013:14th |
| james su | Q3 2010:56th, Q2 2011:7th, Q3 2011:11th, Q4 2011:14th, Q2 2012:15th, Q1 2013:7th, Q3 2013:15th, Q4 2013:15th |
| Chad Lee | Q2 2011:26th, Q3 2011:17th, Q4 2011:12th, Q1 2012:1st, Q2 2012:23rd, Q3 2012:28th, Q4 2012:22nd, Q1 2013:20th, Q2 2013:8th, Q4 2013:16th |
| Jeroen Rutte | Q3 2010:19th, Q3 2012:10th, Q4 2012:11th, Q1 2013:8th, Q2 2013:18th, Q3 2013:4th, Q4 2013:17th |
| Frank Puechl | Q3 2012:26th, Q1 2013:15th, Q3 2013:12th, Q4 2013:18th |
| Rytis Budreika | Q4 2013:19th |
| mentzel.iudith | Q4 2010:4th, Q1 2011:17th, Q2 2011:23rd, Q3 2011:5th, Q4 2011:4th, Q1 2012:7th, Q2 2012:16th, Q3 2012:33rd, Q4 2012:4th, Q1 2013:2nd, Q2 2013:7th, Q3 2013:2nd, Q4 2013:20th |
| João Barreto | Q3 2010:18th, Q4 2010:21st, Q2 2011:3rd, Q4 2013:21st |
| Telmoc | Q4 2013:22nd |
| Randy Gettman | Q3 2010:8th, Q1 2011:25th, Q2 2011:10th, Q3 2011:3rd, Q4 2011:11th, Q1 2012:18th, Q2 2012:19th, Q3 2012:30th, Q4 2012:12th, Q1 2013:3rd, Q2 2013:13th, Q4 2013:23rd |
| Yuri Pedan | Q3 2010:45th, Q4 2010:9th, Q1 2011:2nd, Q4 2013:24th |
| Viacheslav Stepanov | Q1 2011:8th, Q2 2011:4th, Q3 2011:13th, Q4 2011:19th, Q1 2012:17th, Q2 2012:9th, Q3 2012:6th, Q4 2012:21st, Q1 2013:18th, Q2 2013:22nd, Q3 2013:11th, Q4 2013:25th |
| Krzysztof Helbin | Q1 2012:10th, Q4 2012:18th, Q4 2013:26th |
| Anil Jha | Q2 2012:28th, Q4 2012:19th, Q4 2013:27th |
| Kevan Gelling | Q2 2011:19th, Q3 2011:1st, Q4 2011:6th, Q1 2012:6th, Q2 2012:22nd, Q4 2012:16th, Q1 2013:12th, Q4 2013:28th |
| Siim Kask | Q1 2011:28th, Q2 2011:6th, Q3 2011:10th, Q4 2011:3rd, Q1 2012:8th, Q2 2012:4th, Q3 2012:31st, Q4 2012:10th, Q1 2013:11th, Q2 2013:16th, Q3 2013:17th, Q4 2013:29th |
| Leszek Grudzień | Q2 2013:24th, Q4 2013:30th |
| Yuan Tschang | Q2 2012:24th, Q3 2012:24th, Q4 2012:27th, Q2 2013:30th, Q4 2013:31st |
| Niels Hecker | Q2 2010:2nd, Q3 2010:1st, Q4 2010:15th, Q1 2011:6th, Q3 2011:7th, Q4 2011:10th, Q1 2012:2nd, Q2 2012:3rd, Q3 2012:2nd, Q4 2012:7th, Q1 2013:6th, Q2 2013:3rd, Q3 2013:13th, Q4 2013:32nd |
| Oleksiy Varchyn | Q3 2013:28th, Q4 2013:33rd |
| _tiki_4_ | Q4 2011:26th, Q1 2012:16th, Q2 2012:20th, Q4 2012:8th, Q3 2013:8th, Q4 2013:34th |
| Goran Stefanović | Q1 2012:31st, Q2 2012:27th, Q3 2012:34th, Q4 2013:35th |
| Rakesh Dadhich | Q2 2012:10th, Q1 2013:23rd, Q2 2013:28th, Q3 2013:27th, Q4 2013:36th |
| Stelios Vlasopoulos | Q4 2010:37th, Q4 2011:20th, Q1 2012:27th, Q2 2012:30th, Q3 2012:1st, Q4 2012:1st, Q1 2013:27th, Q2 2013:23rd, Q3 2013:19th, Q4 2013:37th |
| Thierry Poels | Q3 2011:22nd, Q1 2012:24th, Q1 2013:26th, Q3 2013:25th, Q4 2013:38th |
| Milibor Jovanovic | Q4 2012:29th, Q1 2013:30th, Q3 2013:18th, Q4 2013:39th |
| Dan Kiser | Q4 2013:40th |
Hi, Steven!
ReplyDeleteIt is quite strange for me:
12 Manfred Kleander (1) Austria 34 mins 52 secs 84% 2393
20 mentzel.iudith (13) Israel 33 mins 13 secs 84% 2251
The correctness of 2 players is the same, but the one who took more time to answer is placed 8 positions higher than the second one who took less time?
Ravshan, you have a sharp eye. The score you see above and on the rankings page is actually the weighted score, which applies a formula involving time, to the "base" score, which is not shown.
ReplyDeleteIf, however, you drill down to this championship via View All Past Competitions (bottom of Take a Quiz on home page), you will see that Manfred has a base score of 618, while Iudith's is 583. This is not a big gap; the #1 winner has a base score of 686.
So your question really comes down to: how can the base score be different with the same %correct?
And the answer is that the total score is a sum of scores of individual quizzes, and the penalty paid for a wrong choice varies depending on how many choices there are in that quiz. So likely Manfred was wrong more on quizzes with lots of choices, while Iudith was wrong on quizzes with fewer choices.
Ah, the joy of algorithms!
Oh, now it is more obvious for me. Thanks for the clarification, Steven!
ReplyDeleteHello Steven, Ravshan, All,
ReplyDeleteIf we look at the ranking list, something might look very strange:
Players with correctness of 70 to 80% are "spread" from Rank 7 to Rank 32,
which means 25 ranking positions out of the 40 participants.
I think that this also somehow suggests that the playing time still "weighs" too much
in comparison with the correctness.
These two factors are also not completely independent, in the sense that the playing time being limited,
you should play faster for making sure that you will be at least able to see ALL the quizzes
before the time expires ... and doing things faster normally means being more error prone,
we don't even need a Playoff to acknowledge this, real life proves it almost every day.
I completely agree that those who are able to be both fast and correct at the same time should definitely
rank higher ... but, especially in the Playoff, a relatively low correctness most probably means
lack of time (mainly for becoming aware to all the small details and differences between the often
very long choices) rather than lack of knowledge.
I am pretty sure that, if instead of having to "decipher" so long pieces of code, the very same knowledge
would have been used for asking the very same question, but formulated "in words only",
probably many players would have achieved a much higher correctness percentage.
For me it is still very pleasant to remember the first 3 Playoffs, back in 2010,
with those much shorter and much more focused quizzes.
It's true that the world becomes more and more complex with the time, but, unfortunately,
the human brain still has some limitations regarding what is it able "to consciously assimilate"
in a very short time.
Thanks a lot & Best Regards,
Iudith
From Viacheslav Stepanov:
ReplyDeleteHello All,
I absolutely agree with Iudith. Playoff time weighs too much.
I would propose one or all of the following:
1) Value correctness more than time, use time to resolve ties only.
2) Make quizzes shorter, more focused and reduce time. For example 10 quizzes, 20 minutes.
Anyway time should not matter too much. There is nothing great in say 5 minutes and 50% correctness. You could click randomly and get similiar results. But now this result could outperform 80% and 35 minutes.
Best regards,
Viacheslav Stepanov
And do you think this different approach should also be used for SQL quizzes and any other championships?
ReplyDeleteMy opinion - for all playoffs correctness should take precedence over time
ReplyDeleteViacheslav
I do strongly agree with Viacheslav, time should be used to solve the base score ties.
ReplyDeleteAlthough, Q4 2013 was my first play-off (I've registered in 2011), I have disappointed to see among questions the one on the feature I have never used before. It was Quiz #4 with a very long setup code, and (omg!) 14 choices! Moreover, about one third of the choices were long like a standalone programs and they were cumulative, meaning that you had to decide not only the correctness of the code, but should decide if it has to be applied. Anyway I'm happy I had a chance to participate in the play-off.
I assume I might be wrong and the rest of the world uses actively RLS/VPD, but my suggestion, in general, is to avoid (or at least try to avoid) very rare to use features for the play-offs.
Hello All,
ReplyDeleteWe once discussed in the past various variants of SELECT statements for ordering the Playoff results
for ranking.
Mainly, the idea was to allow the playing time to be relevant only when the correctness
does not differ by more than "N%".
What Viacheslav does suggest, namely the 10 quizzes in 20 minutes, is exactly the Playoff format
used in 2010, in the first 3 Playoffs.
I agree that any decision regarding Playoff rules will have its pros and cons.
It is hard to suggest an ideal format, that will cover in the best possible way ALL the aspects
involved: number of quizzes, playing time, length of quizzes, topics to be touched or not,
ranking algorithms, and so on.
After many playoffs played, however, I can definitely say that for me, at least,
the main problem was always that of the too long choices / too much code for the given time.
In my opinion, as I have always expressed it, the variant of 10 shorter quizzes has the advantage
of covering more topics and affording a better chance to knowledge to prevail upon the
"fast eye debugging of long code" capability.
For allowing knowledge to be relevant, you should fully comprehend the problem,
and, for me at least, this is very difficult to do when you have to assimilate (or deduce)
a full scenario by just looking at code and required output.
We all know that real life sometimes does require such abilities from a developer,
mainly for correcting bugs,but, even in those cases, the right way to go is to reverse-engineer
the problem, before attempting to correct a buggy solution.
Otherwise, chances are that new bugs will just replace the old ones.
Regarding the other playoffs, it is hard to say yet, because we all have less experience.
However, based on my personal feeling only, considering quizzes complexity versus playing time,
the SQL playoff looked to me very well balanced, while the Logic playoff was far more difficult,
and, in spite of the longest time of all, there was a certain moment when you could not do
anything better but start to guess.
Thanks a lot & Best Regards,
Iudith
I also think correctness is more important then time
ReplyDeleteI think the timing should have less weight than it does now.
ReplyDeletePersonally I would prefer shorter questions as by the time I reach the last line of the quiz, I don't remember the first line anymore and I lose concentration and very often misclick the answer as I have to scroll backward and forward. But it's my personal opinion. Always glad to participate in Play offs though. Makes my quarter. :)
Best Regards
Anna Onishchuk
With a set of Advanced quizzes, as we get for all championships, I think the different base score for the same % correct is as much of an issue as having too much weight given to time. My suggestion for ranking championships would be to have the primary ranking based on number of choices correct (what is already displayed as % correct), with the time taken purely being a tiebreaker.
ReplyDeleteTo me, championships of 10 presumably advanced quizzes in 20 minutes is forcing the time issue to affect the % correct, rather than encouraging players to improve their % correct. They are vague recollections, but I thought that was part of the reason why the format changed.
On the other championships, I thought that the SQL championship was good, but the Logic one needed to use significantly harder quiz formats than the weekly ones because there are the generators for standard quiz format.
Thanks,
Tony
I think we are loosing the focus about the real thing here. The tests with predefined answers have ONLY one advantage: they are very easy to be evaluated. The problem is that any test taker has no chance to express its creativity and way of thinking. For this reason I think that putting a higher or a lower weight to the time is quite unimportant. They are already too artificial to look like "real life" does.
ReplyDeleteHello all,
ReplyDeleteI've just re-read one of the older threads on the same topic to refresh my memory
( see http://plsql-challenge.blogspot.de/2012/11/change-scoring-algorithm-for-playoffs.html ).
Since I've never played one of the older playofss (10 questions in 20 minutes? Sounds ... interesting) , I can't comment
on whether the old system was better or worse than the one we're currently using.
However, regarding the proposal to use time only as a tie-breaker between players with same % correctness,
I don't think that's the right approach; to quote Chris Saxon from the thread mentioned before:
"I think that the scores should still be weighted. If one player manages to get 90-95% in half the time of a player who
got 100% I would rate the first as "better". I can't decide how low the quicker player's correctness % would need to be
before I thought the second better, but it would probably be in the 80s."
It might make sense to decrease the weight of time compared to % correctness, though.
Nevertheless, there are indeed things that IMHO could be improved:
- don't use excessively long questions in the playoffs. At least one of the questions in the last playoff falls into this
category
- if using shorter questions, perhaps increase the number of questions (as apparently was the case
in the old playoffs). Of course, this might not be viable, because it causes additional work for Steven and the
other people who write the championship quizzes.
- it always bothered me that players having the same % correctness might or might not be ranked according to time.
The scoring of the questions / answers should me modified, so that equal % correctness = equal base score.
I find results like this:
9 Ravshan Abbasov (1) Uzbekistan 33 mins 11 secs 79% 2441
10 Anna Onishchuk (8) Ireland 16 mins 40 secs 79% 2432
where two players have the same % correctness and the player who took twice as long (!) gets a higher rank
very, very confusing.
Just my 0.02€
Frank
Hello All,
ReplyDeleteJust to add 1 more penny to those of Frank above:
I think that the "equal base score" can be achieved by ranking ALL the choices of ALL the quizzes
together, that is, as if the entire competition contained "one single big quiz", with all the choices
belonging to it.
As Tony said above, all the quizzes being Advanced level, this looks suitable.
An objection might be raised that some choices may look much simpler in comparison with others.
I think that following Frank's advise of not using excessively long quizzes,
will also render all the choices as "comparable", and then the "scoring together" may work.
By the way, in the 10 quizzes variant Playoff, I remember that part of the quizzes used to be Intermediate level. Anyway, the 2 minute per quiz somehow automatically excluded the excessively long quizzes anyway.
Thanks a lot & Best Regards,
Iudith
Somehow this reminds me of the discussions we have had in my company about bonus calculations for sales people. 8 people involved in the decision having at least 10 different opinions and arguments about "what is fair". We managed to code a dynamic system that allowed manager to setup rules that could handle 6 different "fairness" parameters. But noone could figure out why their bonus was such-and-such and support was swamped trying to explain algorithms. So it was soon stopped and replaced by a very much simpler algorithm - might be a few "unfair" bonuses here and there, but people can understand it.
ReplyDeleteFor the quizzes in the weighted-score calculation it has been attempted to include multiple parameters to be "fair." Naturally that makes it more difficult to understand - just as our bonus system ;-)
So we score each quiz according to how many correctly marked choices out of the number of choices in the quiz. Correctness obviously is most important - fairness parameter 1 :-)
Now should a correctly marked choice in a quiz with 4 longish choices count as much as a correctly marked choice with 14 short choices? What is most fair? Fairness parameter 2 here is that each quiz totally counts equally, so in a 5 quiz championship a correct answer in the first case is worth 1/4 * 1/5 of total base score, while in the second case a correct answer is worth 1/14 * 1/5. This is the reason why two players can have same % correct choices but different base scores - if one of the players had two wrong choices in the 4-choice quiz and the other had two wrong choices in the 14-choice quiz, for example.
And then fairness parameter 3 is weighing the base scores according to time used - which is meant to give higher scores to players who can quickly answer correctly as they probably know their stuff and haven't been spending time googling it. (This parameter can be argued is more important for the daily quiz and less for the championship, as the championship is time limited anyway.)
All this seemingly complex calculation has evolved over time as response to players crying "unfair" and then some rule was invented to try to address each issue and make it "fair". But the price of the fairness is complexity and making it hard to understand why one player gets higher rank than another. You can't have both - you must choose a suitable middle ground thats reasonably fair without being too complex.
My personal opinion is I haven't heard any new arguments that warrants much change to the "middle ground" that is the current set of rules. Only thing I would consider is that for championships specifically I can understand that time factor could be decreased as the championship already has a timelimit to preclude too much googling.
Hello Kim,
ReplyDeletethanks for the explanation. Might I then suggest that - additional to the overall % correctness - the rankings display the "weighted % correctness" (for lack of a better term) / % correctness for each individual quiz? I guess this would make the ranking process much more transparent.
Kind regards,
Frank