28 March 2012

Ah, Those Edge Cases! (11632)

The 27 March quiz tested your knowledge of collections, and the kinds of assumptions you might make when writing a program that accepts a collection as a parameter.

The way the quiz was formulated was different from the usual, and it definitely caused some frustration (one player wrote "Maybe it is because I was awake for about 2 hours last night doing some DBA work for a client, but I did not even understand the question that was being asked. Even after looking in the Library to see the Topic/Summary, my co-worker and myself are still confused on what exactly was being asked." and another: "This question was confusing. I assumed that the requirements were the requirements of the code that was being called. In other words, if all the conditions are met the code will work properly, not does this code enforce all the conditions.").

Our very smart and dedicated reviewers caught many issues with this quiz, and we spent a fair amount of time discussing ways to tighten up the quiz (and I followed almost all of their advice). Yet, in the end, we were "defeated" (that is, an issue was identified in the way the quiz was scored) by an edge case regarding string-indexed arrays - which was covered in the 22 March quiz.

Several players pointed out that choice 9991 (the choice that was scored as correct and stated that no requirements applied) would require an integer-indexed array, since the algorithm would not correctly display all elements in the collection if one of the index values was the empty string.

How right they are, as one player demonstrated with a script:
SQL: CREATE OR REPLACE PACKAGE plch_pkg AS
  2     TYPE my_collection IS TABLE OF VARCHAR2(10) INDEX BY VARCHAR2(20);
  3     index_type VARCHAR2(20);
  4  END plch_pkg;
  5  /
 
Package created

SQL: CREATE OR REPLACE PROCEDURE plch_proc(numbers_in IN plch_pkg.my_collection) IS
  2     l_index plch_pkg.index_type%TYPE := numbers_in.FIRST;
  3  BEGIN
  4     WHILE (l_index IS NOT NULL) LOOP
  5        DBMS_OUTPUT.put_line(numbers_in(l_index));
  6        l_index := numbers_in.NEXT(l_index);
  7     END LOOP;
  8  END;
  9  /
 
Procedure created

SQL: DECLARE
  2     l plch_pkg.my_collection;
  3  BEGIN
  4     l('b') := 'b';
  5     dbms_output.put_line('without null index');
  6     plch_proc(l);
  7     l('') := 'a';
  8     dbms_output.put_line('with null index');
  9     plch_proc(l);
 10  END;
 11  /
 
without null index
b
with null index
 
PL/SQL procedure successfully completed
Another players also wrote to say that "There is a requirement that the collection be non-empty, but no requirement that the collection is initialized (relevant for nested tables and varrays)." This is true. We did not include as a requirement in the list the need for a collection to be initialized. And we did not specify that you should only consider the requirements in the list.

What's a quiz administrator to do?

1. My biggest problem with this quiz is the possible complications for future quizzes. It is one (interesting) thing to know that an empty string as index value can cause odd behavior. It is quite another thing to have to account for this scenario in every quiz on string-indexed arrays. Seems like an unnecessary complication. So I plan to add to the assumptions for the daily PL/SQL quiz that unless otherwise mentioned, all index values of string-indexed collections have a non-NULL length. Any objections to my doing this?

2. Add to this quiz a statement to the effect that only those requirements listed in the question are to be considered. That is, there might be other requirements (such as "The nested table or varray must be initialized.") that could apply, but are not to be considered in this quiz.

3. Give credit for a correct answer on 9991 to all players who submit feedback indicating they marked this choice as incorrect because of the possibility of an empty string index value. I very much doubt that many players recalled this edge case and applied it to this quiz, but if you did, let us know and we will give you credit.

4. Give credit to any player who marked all choices as incorrect and submit feedback indicating that they did so because we did not include a requirement specifying initialization of nested tables and varrays. I say "all choices as incorrect" because if you thought of this additional requirement, then none of the choices could be correct (only the first even includes the "must be associative array" requirement).

Did I miss anything?

3 comments:

  1. IMHO I wouldn't want the assumption added to the default assumptions. I'd prefer it be noted in any quiz that might be affected. I realise this will be more work (to update past quizzes) but I'd rather we work against making the list of assumptions so long that it's impossible to remember them all.

    ReplyDelete
  2. Hello Steven, All,

    I tend to agree with Jeff Kemp in that I would not add that assumption as a default assumption, but rather specify it where it could be relevant, using a sentence like the usual
    "You may suppose that the all the index values are not empty strings".
    Otherwise, what would happen with a nice quiz like that of March 22, where this issue was the central point ?
    In this quiz of March 27 for example, since it deals with a generic procedure valid for ALL cases, an explicit specification like "indexed by a NON-EMPTY string" would be sufficient, just to eliminate eventual objection, and WITHOUT supplying too much information that could have disclosed the choice correctness.
    For the March 22 quiz this is not the case, there any additional information would have disclosed the answer, while making it a general assumption would have made the entire quiz be not relevant.

    The difficulty with this quiz was that the requirement that the code should be working IF and ONLY IF all the specified requirements are met was quite unusual if we consider daily life.
    That is, in daily life you should create code that should work if certain requirements are met ( the IF part above ), but usually no one imposes the restriction that the code SHOULD NOT work if not all the assumptions are met (the ONLY IF part).
    On the contrary, usually the fewer assumptions allow the code to work, the better.

    Regarding the issue of according credit for a choice based on a feedback:

    If indeed an issue arises around a specific choice (like in this case) and you agree, following a discussion like this one, that certain players should be given credit based on a feedback
    that explains why did they make their choice the way they did, then maybe it could be a good idea that, a condition for receiving the credit to be the submission of the explanatory feedback BEFORE the quiz is closed and the results and explanations are published.
    This could answer your doubt at point 3 above.

    Anyway, all in all, an interesting and educative learning experience !

    Thanks & Best Regards,
    Iudith

    ReplyDelete
  3. Good points, Jeff and Iudith. I will not add to assumptions, and hopefully we will manage to remember this edge case in future quizzes.

    Regarding giving credit only to those who give feedback before results are published: that doesn't really work too well. A player could mark 9991 as incorrect precisely because of the empty string case, and think nothing more of it - they are correct, after all! It is not until the results are published that the player sees that we missed the edge case.

    But I have found that players do not "take advantage" of this kind of "offer." Very few will follow up and claim credit for the choice (only one has done so for this quiz).

    ReplyDelete