The PL/SQL Challenge (www.plsqlchallenge.com) offers a daily quiz on the PL/SQL language, through which thousands of Oracle technologists demonstrate and deepen their knowledge of PL/SQL. This blog contains posts by the PL/SQL Challenge founder, Steven Feuerstein, as well as comments from players.
30 November 2010
29 November quiz: interesting reactions to a "tricky" quiz (1706)
The 29 November quiz, the first authored by Ken Holmslykke, one of my invaluable quiz reviewers elicited several interesting reactions. It was a tricky quiz, no doubt about that. Involving implicit conversions and comparisons of string values, it was a hard one to sort out.
I received the following two observations from players:
1. This is not an objection to the quiz at all (I did get i right , this is more a slight concern about "question styles..." Three cheers for Ken (nice guy taking some of the load off Stevens shoulders. But his "question style" is somewhat different. The thing is, that the "topic" of the question mostly is what happens with the implicit conversions of numbers to strings. But the use of all_objects.object_type%TYPE in my opinion mostly serves to confuse the reader so he won't recognize what the question is really about. Had the subtype index_t been declared as VARCHAR2, the question would have been clearer, that the subject was to test the readers knowledge of how a string associative array works if you use numbers working with that array. So just a slight warning (mostly to Ken) not to try to be too much "clever" when formulating questions. PL/SQL Challenge is mostly about learning, which to me means that it should be reasonably clear what topic the questions are trying to test our knowledge about. This tricky "obscuring" of the question topic is not (IMHO) consistent with the normal "question style" of Steven. That said, I do believe it will in the long run be a good thing to have questions from different authors, who might have a different angle than Steven Just my 2 bits.
2. It's a first time when I really want to nominate this question as a "Question of Year" (or at least a Quarter). I choose a wrong option, but I don't regret. It took a while for me to explain this right option. The only concern I have is about NLS_SORT / NLS_COMP parameters. People will definitely raise this point up, because you even may create your own character set definition and specify a different sort order using those parameters. I hope it's hard to achieve it and and question will be left unmodified.
Regarding #1, ironically, though Ken wrote the original quiz, I did some editing and I am the one who introduced the use of the subtype. Ken originally declared the type as VARCHAR2. So I must take the "blame" for that obfuscation.
Regarding #2, we document in the assumptions a default installation of an Oracle instance with a specific character set. Is that not enough to cover the issue raised regarding NLS_SORT and NLS_COMP?
We'd love to hear what you thought of the quiz.
Subscribe to:
Post Comments (Atom)
Yes, this was really tricky one :)
ReplyDeleteComparing with previous quizzes (made by Steven) where I haven't seen any (real) trick to confuse players.
I did it wrong. Not because I didn't notice string type of index. I did but I though that the quiz was about the fact that '1'+1 returned number 2. And after taking quiz I tried it and was very very surprised when I saw 10 as output. I took me about half an hour to figure out how this was possible.
Very tricky :) But very nice. I liked it after all. It was very challenging and told me to pay even more attention to all aspects of code in question.
Yes, I liked it.
Regards
Ludo
Tough question! I knew the 150 was going to be the wrong answer but couldn't work out why... so I chose it anyway.
ReplyDeleteMy only advice would be to provide more potential answers to a (slightly obvious) trick/tricky question. I almost chose the right answer because there weren't enough potential wrong answers to throw me completely off the scent.
I confess I'm torn about the subtype issue. On the one hand, I agree with the feedback that it introduced some confusion as to what was going on and make the quiz even harder than it would have been.
ReplyDeleteAnd yet, "hard-coding" a type is generally not good practice if it can be avoided with subtypes, so shouldn't the quizes also reflect best practices?
Overall, I think this was a very tricky way to test knowledge of the character vs. integer behavior. Maybe it would be better classified as an "advanced" quiz?
For most of us, the SUBTYPE declaration probably didn't add much difficulty to the quiz; however, it also didn't add any significant value. The only benefit of such obfuscations is that it requires players to take a bit more time reading the quiz before responding which should make it easier to identify those who are attempting to cheat their way to the top. It is astonishing to see that some players responded to the quiz 100% correctly in just a few seconds. It is extremely unlikely that anyone could have reliably read and evaluated the code in so short a time.
ReplyDeleteMy apologies to Ken :-)
ReplyDeleteAnd don't misunderstand me - I think it is a great question to learn from. I'm just concerned that readers spend more time trying to work out what the question is really about (string associative arrays), than they do excercizing their knowledge about associative arrays.
I may be wrong ;-)
Regarding the difficulty level: it is, of course, hard sometimes to choose the level. In this case, I decided on intermediate because while it is tricky, the actual concepts in PL/SQL that determine the correct answer are not terribly advanced, right?
ReplyDeleteSteven, I agree, leave difficulty level as it was (intermediate).
ReplyDeleteI also raised the decision on the level of the question. And yes the subject is intermediate. But the way that the question is asked, makes it advanced.
ReplyDeleteBut it is indeed very important to take a good look at declarations. Oh well the level is not important (makes it any difference in the scoring? I guess not.) it is the subject that was very interesting.
As crikeymo said, there should have been more choices. It is possible that some of the faster correct responses were due to players guessing because it looked like a trick question (I still don’t believe a 4 second correct response is legitimate).
ReplyDeleteI agree with maase: when practical, the quizzes should exhibit good coding practices. There are times when undesirable coding practices are necessary because there is no other way to adequately demonstrate the topic of the quiz or because following good practices would make the quiz needlessly verbose. In this case the use of a subtype did not demonstrate good programming practice; there is no reason for the index type to be based on the column type except to intentionally make the code less obvious (which is definitely a bad coding practice).
I was very pleased to apply the knowledge to this excellent ADVANCED question.
ReplyDeleteSteven said "while it is tricky, the actual concepts in PL/SQL that determine the correct answer are not terribly advanced, right?"
Of cause. It's more hard to call trivial count method of collection, isn't it? :)))
Wow !!!
ReplyDeleteThis was indeed a VERY, VERY, VERY nice question,
I really loved it !
What is interesting is that I practically browsed the scores of this quiz up to the end of the list and found that many players in the first part of the list had a relatively high response time (about 200 seconds or more) but achieved 100% correctness,
while those close to the end of the list had much faster times (about 50 seconds) and achieved 33% correctness.
The piece of code was "short enough" for those who might have decided to effectively type in the code and test it before choosing the answer,
and this typing time would probably have been sensibly equal to the time taken by others "to CORRECTLY compile (and RUN !) the code with the eyes only", as we usually do, and choose the right answer with no testing at all !!!
I would be extremely curious to know what is the percent of those who really answered without testing out anything !!!
I wish "some tool" were available that could allow for making statistics on this aspect,
a different one (of course) than creating extremely long code, that would indicate readily from the start that trying to type the whole of it IS NOT the way to go ...
For this quiz, maybe more than for any other one that I have encountered up to now,
"reading only" the code and answering correctly
would really demonstrate good knowlegde, in my opinion.
It is so pleasant to see that with so few "preparation pieces" it is still possible to raise up such "tricky ones" that just await you behind the corner ...
Oh, those small jewels born out of "micronic gold particles" and able to shine upon our lives !
This quiz was a really, really exciting experience !
Thanks equally to Ken and Steven !
Best Regards,
Iudith Mentzel
In short, I'll vote the way of limiting the "trick" of questions such as this.
ReplyDeleteA subtype definition I think would be appropriate in this case. While I always check the datatype definitions and I realised in this case I needed to look harder - this was a little too obfuscated, IMHO. The other clue that set me off was - why would you use that column for a %TYPE definition?!
Also, sometimes I use the difficulty level to determine how much thought I should put into the question. If it's beginner, I know I just need a quick run through syntax and try to answer quickly.
And that is why I actually feel that the difficulty level should not be displayed along with the quiz question - until after you have taken the quiz. I don't even remember asking to have it put there. Hmmm. My developers are a bit too independent or I have forgotten what I asked for. Perhaps both.
ReplyDeleteNo! Don't hide the difficulty level :) I use it too - it comes in especially useful when the question involves a large amount of code. It means I don't have to look for very subtle nuances, such as when a subtype is based on ALL_OBJECTS.TIMESTAMP%TYPE...
ReplyDeleteSteven, I agree that the difficulty level should not be displayed when the quiz is presented. Advertising the difficulty level probably alters the way players respond. One may attempt to answer Beginner questions quickly since the time penalty may be more deleterious than a single incorrect choice. Removing the difficulty hint would likely result in scores more reflective of players' PL/SQL knowledge than their ability to estimate and manage risks.
ReplyDelete@jhall62
ReplyDelete"It is extremely unlikely that anyone could have reliably read and evaluated the code in so short a time" - I did it :)
Not related to this question - but would be nice to have for future playoffs. A possibility to skip quizes and return to them late (if time permits).
ReplyDeleteKeep the tricky question coming. It's those kind of question that make me come here everyday ! If all the question where so straigthfoward, I could just read the documentation (RTFM !!!) !!!
ReplyDeleteThank you for your comments. I appriciate them.
ReplyDeleteThe background story of the quiz is, that it is one of my own code problems. I had a package where I used the id as index in an associative array and one day we saw that the sequence would pass 2^31.
The solution for this problem was to alter the associative array from index by pls_integer to index by varchar2(). This turned out to work very well.
My problem was that my proces also had a compression algorithm where logically similar rows were combined and assigned a new id with the index l_list(l_list.last + 1).
In stead of just writing a memo that probably wouldn't be read by anyone I decided to submit it to Steven. (After all it was one of Steven's sessions that first introduced me to the index by varchar2 possibility). It was a behaviour that I didn't think of when correcting my first problem, didn't show in my first early test (to few rows selected), and afterwards I thought I'd learned something new.
I tried to construct the question in a way where the difficulty level (Intermediate), topic (Comparing Contents of Strings) and answer options would indicate that it is not straight forward but at the same time not telling so much that the quiz would be forgotten immediately.
> crikeymo, November 30, 2010 2:45 PM
> provide more potential answers to a
(slightly obvious) trick/tricky question
We considered it. In this kind of question most players will see that at most one solution can be correct. So if we give e.g. 6 options and you select one random reply you will get four correct replies.
> jhall62, November 30, 2010 3:42 PM
> the SUBTYPE declaration probably didn't
> add ... any significant value
The subtype was there because in my original code it was declared in pks and the code in a pkb. I copied it without thinking. Bottom line: You are right.
> Kim Berg Hansen, November 30, 2010 3:50 PM
> I think it is a great question to learn from
Thank you. That was actually my primary objective.
> readers spend more time trying to work
> out what the question is really about
The only way I can see this implemented in a quiz like this is to write a different text in the topic line. I will remember your comment.
Hello All,
ReplyDeleteThe comment of Jhall62 above:
"
Removing the difficulty hint would likely result in scores more reflective of players' PL/SQL knowledge than their ability to estimate and manage risks."
puts it right and straightforward.
I also think that the difficulty SHOULD NOT
be displayed while the quiz is open.
Hope this opinion will be accepted.
Thanks & Best Regards,
Iudith