Several players wrote with a complaint about their scoring for the 1 February quiz. Here's a typical comment: "Hi, I got a 0 score for February 1st, is this supposed to happen? I answered that none was correct - though I have not found that there was a correct answer I successfully saw the other 3 as incorrect. "
With 1.9 of PL/SQL Challenge, that is as of 31 January 2011, some quizzes are "one choice only can be correct". This happens when the choices offered as answers are all clearly mutually exclusive. In this case, you can only check one box when you submit your answer. That one box might be "No choices are correct."
If you try to submit an answer with more than one box checked, you will see an error message.
Scoring for such quizzes is done on an "all or nothing" basis. If you checked the right box, you get 100%. If not, you get 0%. That is, there are only two different scores you can get, but your weighted score will vary based on the time it takes you to submit an answer.
The PL/SQL Challenge (www.plsqlchallenge.com) offers a daily quiz on the PL/SQL language, through which thousands of Oracle technologists demonstrate and deepen their knowledge of PL/SQL. This blog contains posts by the PL/SQL Challenge founder, Steven Feuerstein, as well as comments from players.
yesterday I chose the option "no choices are correct" and this morning i got a 75 percentage correct.
ReplyDeleteYes, several players have remarked on this issue, as well. We need to either tweak the way this information is shown or provide more of an explanation.
ReplyDeleteFirst, I need to distinguish more clearly between score and percentage correct.
Your SCORE is either 0 or the maximum score possible (depends on the difficulty level). That is, you do not get partial credit for a partially correct answer.
Your PERCENTAGE CORRECT, however, could be 0 or 75 or 100.
Steven asked me to post my comment (sent via mail):
ReplyDeleteSteven,
One free thought I had after yesterday's quiz: somehow it feels like the TIME component of the quiz does not weigh very high, except when you are completely wrong:
I had 0 points (I know I had it wrong, no surprise, I chose 'None of the above'. I did not look well).
If you take a 1000 seconds to have a 100% correct answer, you still have 165 points...If you answer "None of the above" within 94 seconds (which is a reasonable amount of time to answer a question this size), you have no
points (zero):
1. If the answer "None" is incorrect, you get no points at all? It almost looks like it (because my percentage correct is "75%", but I have 0 points)
2. In 1000 seconds you have plenty of time to open an editor, type and run the code (this particular one can be done in less than 200 seconds I think), you still earn 165 points
My lesson: I better take far more time than I do now and type and run the code than to guess by just reading it. But that can hardly be the desired way.
Toine
Steven,
ReplyDeletethanks for the explanation...
I feel much like Toine.
Sterol Andro
As well as tweaking how the results are shown, I think it would be good to highlight to the player taking the quiz that this is an all or nothing quiz as far as score is concerned - especially because the same quiz would have been displayed the same last week (on version 1.8) but scored very differently. This is assuming that there wasn't something to highlight this that I just missed.
ReplyDeleteGood idea. The quiz page will now declare the following right above where you pick your choices:
ReplyDeleteCheck the box next to the choice you believe is a correct answer to the question above. If you believe that none of the choices are correct, check the box at the bottom of the page next to "None of the choices are correct." You can only check one box. Note: you will not receive partial credit for this quiz. If you do not choose the correct answer, you will receive 0 points.
Do you think this will do the trick?
SF
Response to Toine's comments:
ReplyDelete1. If the answer "None" is incorrect, you get no points at all? It almost looks like it (because my percentage correct is "75%", but I have 0 points)
2. In 1000 seconds you have plenty of time to open an editor, type and run the code (this particular one can be done in less than 200 seconds I think), you still earn 165 points
The way the scoring algorithm works today, the lowest weighted score you can get is the "raw" score. So, yes, you could sit back relax and write up all the code you wanted, make sure you get it right, and then be sure to at least get the minimum.
Honestly, I don't see anything wrong with that. If someone is going to go to that effort, why shouldn't they get a minimum number of points?
Does anything feel differently?
SF
Hi Steven,
ReplyDeleteNo no, fair enough. It is me not being aware of all the fine grained rules in the challenge: it is probably written down in the Rules section, but I was not aware that there was at least a minimum score when you have the correct answer, no matter how long it takes. I kind of expected that there was also a kind of maximum time to even earn a positive score (I am a track and field guy, in a decathlon there are certain minium achievements before the score even starts). Everyone can get the correct answer if you look and try long enough. So I learned once again...not a PL/SQL feature, but a scoring rule. I should review the scoring algorithm once more and adapt my strategy when I am not sure about a question. And not rush myself into an answer when I am already looking for, say, 200 seconds.
OK, I need to get myself back on 'track' again after the drop in score with this quiz. It is still great fun! Keep up the good work.
Toine
Thanks for the quick addition of the all or nothing "warning" to players taking the quiz. You had me wondering whether I had simply missed that on the 01-Feb quiz when I saw it on the 03-Feb quiz :)
ReplyDeleteHi,
ReplyDeleteto be honest I never understood that scoring algorithm and actually I do not care much ;) For me it would be enough if you give a small price on random selection, because for me this is more a learning thing than a challenge. On average I miss maybe 1 quiz per week and with that you have already no chance anymore to get into the top list. And at least once per week someone calls or interrupts while I am trying to answer the quiz :D
Regarding the "only one answer correct" thing:
For those things radio-buttons have been invented ;)
Steven, thanks to you and tour team for all the efforts you put into this. I really learned a lot here! Please keep it simple and people should not argue too much on how to interpret certain words. The essence is what is important ;)
David.
> "For those things radio-buttons have been invented ;)"
ReplyDeleteBut it is A P E X - a very, very, very restrictive thing by itself :)