12 October 2010

Correction for 11 October quiz - and change in policy regarding mistakes (1504)

The 11 October quiz contained a mistake - for the first four hour's of its life. I had neglected to append a ";" after the forward references in the choices, with the consequence that none of them would be correct. In the past, my policy had been this: once the quiz is published, it cannot be changed until the day is over. I have decided to change this policy. I will now fix a mistake as soon as it is discovered, usually very early after the quiz starts. That way, for the 1300+ developers who do not play the quiz in the early hours of the day (UTC time), there will be no mistake. And for the relatively few players who did play the quiz with the mistake, you will receive a correction to your score for all choices that were affected by the mistake. I did, in fact, add the ";" to the statements for the 11 October quiz and the scores of roughly 57 players will be corrected to reflect this. Chris Roderick wins an O'Reily Media for being one of the players who brought this problem to my attention. That wasn't the only concern raised by players. The question on 11 October asked, in part: 'Which of the choices offer substitutions for the /*PUBLIC*/ and /*PRIVATE*/ comments so that when the following block is executed, I will see "-100" (without the double quotes) on my screen?' In fact, the choices scored as correct each displayed -100 twice. Some of you felt that therefore all the answers were incorrect, others simply felt this was ambiguous. I disagree. You will definitely see "-100" on your screen. I do not specify that it is the only thing you will see. So there will be no change in scoring for this issue. Cheers, SF

19 comments:

  1. Correcting the option's immediately . . . That's really a good go steven. More over that is fare enough as well. I really appretiate the way the policies and the rules are revisited on a daily basis to make the players life easier and comfortable.

    ReplyDelete
  2. Thanks, Jay. Though I wish I didn't need to revisit my rules on an almost daily basis. Whew. Very time consuming!

    But I do feel that overall the quiz is getting better and better. I hope the over 1300 developers who play each day feel the same way.

    SF

    ReplyDelete
  3. "You will definitely see "-100" on your screen. I do not specify that it is the only thing you will see. "

    Let's say -10000 is displayed. Do you think it's still the correct answer? You do not specify -100 is the only thing in a line!!

    ReplyDelete
  4. Absolutely. If one of the choices had resulted in "-10000" being displayed then that would have been a correct answer (and a really badly-written quiz).

    ReplyDelete
  5. Well, this is becoming more like a tricky language challenge than a plsql challenge. I have to say I'm quite disappointed.

    ReplyDelete
  6. Hi Steven,

    I am writing this just for argument sake :)

    For the Quiz of 20th Sept, the correct choice was

    1
    Mouse
    100
    Keyboard
    Keyboard
    100
    Keyboard

    However there was a 'wrong' choice that outputs the below:

    1
    Mouse
    100
    Keyboard


    Going by your explanation, shouldn't this be treated as correct choice !? ;)

    ReplyDelete
  7. About changing the quiz early for error. There a two sides on this. Most users get a correct quiz, which is nice. But now again it is a good idea to rise early and do the quiz for getting those prices in detecting errors and have a bigger change. (you recently decided to give the price to one of the people who detects errors randomly).

    Hard decision.

    I think that if you display something on the screen, you must be exact in describing it. So if it is possible you see twice -100, the sentence should say "you see -100 at least once", otherwise it is playing with language and the interpretation of it. There are a lot of assumptions that describe the possible differences the different environments in which you run the code (begin Toad, SQLPLUS or other software)

    ReplyDelete
  8. Steven,

    I understand your arguments about the "-100" twice objection.

    But as in the example given by Spoon, I do feel that you are slightly deviating from past practice. Usually (at least that is my feeling) your policy has been that the desired output given in a question should be literally and precisely replicated by a given answer to be correct.

    As Spoon, I am also just bringing the point up for arguments sake :-) I'm not asking for rescore (even though I'm one of those debugging my way through the code in my head and realizing, "hey, this code will always output -100 twice, so obviously this is a trick question with no correct answers - yes, I'm good, I double-guessed Steven" ;-)

    ReplyDelete
  9. Hi Steven,

    When answering the quiz, I knew that the two options displayed "-100" twice, and therefore I did not mark them as correct, because "-100" twice is different then "-100" once, in my opinion. I'm a bit disappointed there will be no rescoring for this ...

    Regards,
    Rob.

    ReplyDelete
  10. When you write code and is told that the correct output should be
    -100

    and then the output is
    -100
    -100

    I would treat that as a bug and therefore try to fix the code.

    This is just ridiculous, please say that you are joking...

    Am currently considering not participating in the final for Q3 and abandoning PL/SQL challenge all together.

    Regards
    Johan MÃ¥rtensson

    ReplyDelete
  11. He Steven,

    I guess that the expected output should be specified more precisely.

    From my point of view you should add either "at least" or "and only it" explicitly (or add an appropriate assumption into the quiz assumptions).

    Regards,
    Oleksandr

    ReplyDelete
  12. I agree with xuhui and Spoon. Few weeks ago you rescored a quiz where only difference in output for two different choices was a new line character (sqlerrm vs dbms_utility.format_error_stack). I think the question like "when executed you will see.." should be more clear, e.g. "that's the only thing you will see" or "this is more or less what you can see".

    Of course, I understand that this is supposed to be about PL/SQL knowledge and skills and not about tricky questions so many people ignored yesterday the fact that they would really see -100 twice as the question was about forward referencing. That's why I was surprised that you agreed to rescore quiz with a new line character in output I mentioned, I though that someone just wanted to be too clever (pointing a new line character which didn't change meaning of the quiz at all).

    So I believe you should be more consistent in judging this kind of "mistakes".

    Regards
    Ludo

    ReplyDelete
  13. I thought this was a quiz that tests one's knowledge of PL/SQL, and not one's ability to detect tricks and clever play of words. Was it that hard to simply ask which of the choices results in -100 being output twice and avoid ambiguity?

    ReplyDelete
  14. I think that allowing the result to appear twice as valid is really unfair.

    I would be willing to bet that almost everybody who (like me) said there were no valid choices did so entirely because the two which have been marked as valid do not only display "-100".

    That is just WRONG.

    ReplyDelete
  15. Hi Steven.

    I do feel this is not 100% to put such tricks into the questions.

    You have asked to see "-100". OK.
    But in this question you would see
    "-100
    -100"

    I understand it may be difficult to admit that the question has been formed badly -- I have this myself and will always do any logical trick that proves that I had been right, no matter what.
    But the fact is that in previous quizzes you have been specifying in detailed way what is supposed to be on the screen, including the multiplicity of the results.

    I do agree with xuhui above -- please, to not change this quiz into Tricky-English Challenge, as we are not all English native speakers. Keep it PL/SQL Challenge.

    ReplyDelete
  16. @xuhui: Gimme a break - in yesterday's quiz I didn't find any challenge concerning the English language. And I am not a native English speaker.
    The "-100" is displayed twice, so what?
    I agree with Steven that "You will definitely see "-100" on your screen".
    And I am sure Steven wouldn't do that "-10000" thing you've constructed, although I'd neither agree on "-10000" being correct in this case. But that's a rather academic discussion, no need to get disappointed in my humble opinion.

    ReplyDelete
  17. Well, if you take it that far, here is the complete works of Shakespeare:

    ABCDEFGHIJKLMNOPQRSTUVWXYZ

    (some letters have been rearranged, and duplicates removed)

    jk ;)

    ReplyDelete
  18. PL/SQL developers of the world, unite!

    The message is coming through loud and clear: I was wrong and you are right.

    Yes, this was ambiguous. Yes, it is inconsistent with at least one previous quiz (thanks for digging up that quiz, Spoon!). Yes, I need to be completely unambiguous and not play "tricks."

    I will change the quiz question to specifically ask for those choices that display -100 twice. I will give everyone credit for those two choices that recommend a forward reference of proc2. I will recalculate scores and ranks.

    Thanks so much for caring enough about the PL/SQL Challenge (and your rank) to keep me in line!

    Cheers,
    Steven Feuerstein

    ReplyDelete
  19. Hello Steven,

    Maybe having public_proc contain a call to proc1 only, without the call to proc2
    would have spared the output ambiguity ?

    Of course, as an alternative to specifying the right output in the quiz itself as being "-100" displayed on 2 subsequent lines.

    Any of these alternatives would still have preserved the learning purpose of the question,
    which was clearly to check the players knowledge regarding forward reference.

    I really appreciate your endeavour to make the quiz better and better, though it is probably not very easy to invent new puzzles every day.

    Thanks & Best Regards,
    Iudith

    ReplyDelete