08 July 2010

Linguistic confusion on the 7 July 2010 quiz(1248)

Several people have complained about the way this question was phrased. First, to recap, we showed some code and then asked: "Which of these choices contain exception handlers that will cause a compilation error in the block when [handlers] is replaced with the code in that choice?" We then give these instructions (for every quiz):
Check the box next to any choice you believe is correct. If you believe that none of the choices are correct, simply press the Submit Answer button.
One player wrote: About yesterday's quiz, the indications in the text asked to indicate the wrong exceptions, but the column header on the answer table asked for "Correct". This puzzled me, and in doubt, I assumed that the table header was the right thing to do, and I did not check any answer, because none of them was actually "Correct". Today I see that my choice was the wrong one, but I think that the indications in your quiz pages are ambiguous: you cannot set the header "Correct" to indicate a "non-Correct" syntactical construct! And another: Hello, First of all thanks a lot for the idea of PL/SQL quiz. It really inspires me take part in it. Let me comment the quiz published on 7th July (yesterday). I think the question was very confusing and I understood its meaning in absolutely opposite way. I am not English native speaker (I reckon most of people here are not) and that's why I didn't catch the gist of that quiz. Could you please avoid such tricky combinations in future. We are just Oracle specialists and not English language experts. :-) Another person reported that we scored his answer entirely wrong, but I am pretty sure he experienced similar confusion. So, first of all, our apologies for causing this confusion. There can, in essence, be several layers of "correct" that you have to deal with when answering a question:
  • Is the code correct or does it have errors?
  • Is the choice a correct answer to the question?
Having said that, the question clearly asks you to pick all choices that cause a compilation failure. So a choice is correct if it would cause a compilation failure. Of course, I can make all the arguments I'd like to show how, ahem, correct I am - but that doesn't matter if our players are getting confused. We avoid using the words "correct" or "incorrect" in our questions to avoid "overlap" with the usage of "correct" for the choices. That is not, apparently, always enough. We hope to minimize this confusion in the future by changing our instructions to:
Check the box next to any choice you believe is a correct answer to the question above. If you believe that none of the choices satisfies the question, simply press the Submit Answer button.
Now we explicitly ask you to determine correction in relation to the question. Please let us know if you think this helps. We plan on implementing this change in the 1.4 release this weekend.

2 comments:

  1. Perhaps the question could have had the same impact worded this way:

    "Which of the following are true:"
    "1. This would raise an exception: [code]"
    "2. This would cause a compilation error: [code]"
    "3. This would compile without error: [code]"

    In other words, expand each option to be a self-contained assertion that will be either true or false.

    ReplyDelete
  2. Excellent idea, Jeff. I will try to do that from now on:

    Make each choice a self-contained assertion.

    ReplyDelete