10 July 2010

Mistake confirmed in 9 July 2010 quiz(465)

Sadly, the quiz for 9 July had a mistake in it. In the question text, the header of the function looks like this:
FUNCTION nothing_lower (collection_in IN DBMS_SQL.NUMBER_TABLE)
but all the answers had this END statement:
END lowest_index;
Consequently, none of those function bodies would compile and thus none of the answers were correct. Now, of course, this was not my intention, and I expect that many people who took the quiz focused on the issue at hand: the method used to get the lowest defined index value in a collection. Putting aside my careless error (which I will explain below - it is a classic programmer mistake), one of the choices was supposed to be correct:

   RETURN collection_in.FIRST();
END lowest_index; 
We will, therefore, (a) fix all the code so that this mistake is erased from the repository, (b) change all answers on that choice to correct, and (c) re-score and re-rank. So if you marked that choice as incorrect, your score will increase, and so may your rank. This change will not be reflected in your automatic email reporting on today's quiz.

What made this mistake especially infuriating for me (Steven) is that I had just finished cleaning up the question and choices. It all looked good. And then I decided to change the name of the function in the question from lowest_index to nothing_lower. Did I go back and test all my code after making "one little change"? NO! And it blew up the quiz.

I get frustrated when this happens, and I am sure that you do, too. You play the PL/SQL Challenge to test and improve your knowledge of PL/SQL; not to be on the lookout for trivial mistakes or what seem to be "trick" questions. I feat that if we keep publishing quizzes with mistakes, eventually you will give up and spend your time elsewhere.

So...I am going to institute some changes regarding the review and approval process for quizzes. First and most important, an excellent Oracle technologist with whom I have worked over the years has volunteered to review the quizzes. Wow, am I happy! Having another pair of eyes is so important when writing, well, anything, but most definitely when writing code. This step alone should greatly decrease the chance of a mistake sneaking out to you.

Second, I am going to start following a strict checklist for every quiz. Here are the current items on the list:

  • Ran all code, verified that there are no unexpected errors or behavior that differs from what we claim in the question.
  • Each choice is a self-contained assertion. Avoid ambiguity with question and "correct".
  • Is not a "trivia" question, but actually tests knowledge of the language.
  • Does not contain double negatives or complicated text that non-native English language would struggle over.
  • Does not contain "absolute" words like "never", "always", "none", "impossible", etc. They almost always (!) lead to trouble, because so many of you are excellent at coming up with "outlying" scenarios that can provide an exception to almost any hard-and-fast rule.
In other words, among things I hereby promise to you that I will never publish a quiz before running every single piece of code that appears in the question. These steps do not guarantee that we won't make mistakes in the future, but I intend to turn them into extremely rare occurrences. If you have any other suggestions for how I can avoid mistakes or if you know of someone who might want (and be qualified) to review quizzes (the downside being you then cannot play the daily quiz), please let me know. Please post a reply on this blog or send an email to steven@stevenfeuerstein.com.

1 comment:

  1. Looks good. Thanks for making this quiz and still raising the quality of it.