12 October 2011

Typo in Question Text Requires Score Adjustment (7988)

For the first 11 or so hours of 11 October 2011, we asked players to pick a block of code that would display this string:
1*October*2011*12:01:01
As a number of players noted, none of the choices would do the trick - and they wondered if we'd made a mistake. Yes, indeed, that was a typo. So at 6:15 AM Chicago time, Steven woke up, checked his email and was immediately at all the bug reports.

Then he changed the question text to:
1*October*2011*12*01*01
so that the several hundred other Oracle technologists who would be playing the quiz on the 11th would have a more interesting experience.

As a result of these actions, we will issue a correction to the 189 players who chose "Not correct" for the fourth choice (which should have been correct).

It is, of course, frustrating to everyone when we make mistakes like this, and I (yes, this is me, Steven, talking earlier in the third person) apologize to all. But when I am done apologizing, I like to think about how mistakes like this can happen.

You see, I was especially frustrated this morning because at about 03:00 on the 11th (10 PM Chicago time), I received an email from Jeffrey Kemp, one of the most diligent and expert of our players who also happens to live in Perth and so he plays very early in the day (UTC-timewise). He reported this problem. So I took a look. I looked at the question text. I ran the code for the "correct choice"...

And it looked just fine to me! And so another 8 hours passed, and hundreds of other answers submitted, before I took a closer look and recognized/accepted my error.

How could this be? Because we humans have a tendency to see what we want or expect to see, rather than what is really there. This "quick and dirty" approach to life can be helpful at times, but when it comes to analyzing and debugging code, it is a downright nasty tendency.

Professional proofreaders scan text backwards so that they will not be able to read the text and thereby make all sorts of assumptions, and skip over problems in the text. We programmers can't read our code backwards, so we have to make an extra effort to force ourselves to really look at, really read, what we've written, and make sure it makes sense.

That's what I am going to do from now on. Promise.

6 comments:

  1. Don't be too hard on yourself; this happens to everyone, even the most "diligent and expert" of them :)

    ReplyDelete
  2. Hello Steven, All,

    You are right in saying that "we humans have a tendency to see what we want or expect to see",
    there exist even psychology tests that prove how a text with missing or reversed letters or other typo mistakes is read however CORRECTLY as long as the first and last letters of each word are in place, and the word length is about as expected, and this is because of how the human brain works :) :)

    This is the first reason why code should always be tested and it probably is, while the text parts are verified by human readers only.
    Does this mean that we, humans, are less good than machines ?

    Judging by the light-speed answer time of some players, and without considering the network speed itself which can also be a factor, I have already come to the conclusion that probably some players DO "READ" the quiz text and code in such a way, that is, "reading in big" and not word by word or character by character, for good and for bad.
    Otherwise, those answer times are practically unexplainable, they are even lower than the time required to display the page ...


    Back to the quiz itself:

    Just for complete correctness, the explanation currently given for choice [8226]:

    "FM strips out spaces and FX removes leading zeroes, so we see 1*October*2011*12*1*1"

    is not correct.

    It is the FM modifier that strips out both spaces and leading zeroes, and not the FX,
    which has no effect here.

    This is correctly emphasized in the explanation for the other choices and in the Overall Answer.


    And, last but not least, I take your above promise very seriously :) :) :)


    Thanks & Best Regards,
    Iudith Mentzel

    ReplyDelete
  3. Thanks, Iudith. I will fix that explanation. When I first wrote the quiz I mistakenly believed that FX could be used with TO_CHAR - the doc is actually a bit problematic in this regard. My reviewers corrected that, but I didn't fix all the explanations.

    ReplyDelete
  4. Steven is so kind to his reviewers. I think at least one of those monkeys should have caught that error 8-)

    Mike

    ReplyDelete
  5. In case anyone wonders why I would approve such a nasty comment, Mike is one of my fine reviewers (aka, "monkeys"), so he's obviously taking a poke at himself.

    Actually, I am thinking that I might have made a last minute change, thereby undercutting the fine review jobs.

    ReplyDelete
  6. Hello Steven, All,
    Yes, of course the reviewers do an excellent job and I personally do have a very pleasant experience working with them on my quizzes :)

    And, we, the players, are all here to still improve things and, as everybody already knows, we do not forgive mistakes easily :)

    Happy to hear that Steven himself also has to learn things sometimes, that sounds encouraging
    for us, the mere mortals ...
    Yes, the documentation is not always very explicit about everything, can this be expected at all ?

    I don't want "to mix" here the topic of the quiz of 12 October ... that automatic lock removal is not specified in the documentation at all, and I definitely would have expected to find it there if it is indeed a deliberately implemented feature a not just a side effect
    of some other factor ...

    And, just to end with some fun, the monkey is a very sympathetic creature, so no one should feel offended by such a comparison, especially when it comes from inside the team :) :)

    Thanks & Best Regards,
    Iudith

    ReplyDelete