A player wrote the following message to me today:
"The answers for the 02 August quiz are not based on a real Oracle PL/SQL knowledge, but on visual memory abilities. I knew exactly that Sys_refcursor was a weak cursor, but I had rarely used it and I was not sure if the underscore character was supposed to be used there. Moreover, I am sure that if it is a mistake I should not have two mistakes 1) for choosing the option without underscore and 2) for NOT choosing the option with underscore. Assuming that I've made a mistake with underscore character, I don't want to lose 40% of my rate for 1 mistake. that should be 20%. Anyway, I don't think that such answer options really reflect the PL/SQL level of participant, because participant got the gist of question and replied correctly in general, except ""grammar" mistake. In this sense even this one mistake should not be counted as a mistake."
Here's my view on this: the PL/SQL Challenge is intended to test your knowledge and familiarity with the PL/SQL language, and to motivate you to deepen that knowledge and increase that familiarity. Sometimes the question will focus on the way a feature can be used. Somtimes the question will concentrate on "little more" than syntax.
I don't agree with this player, though, that choices that test your familiarity with syntax is not "real" PL/SQL knowledge or simply a matter of "visual memory abilities."
Knowing that the name of the pre-defined weak REF CURSOR type is SYS_REFCURSOR means that you don't have to look up the correct spelling. The more quickly and accurately you can recall syntax, therefore, the more productive you will be.
And, clearly, if you want to make sure not to lose points for a dull issue of correct syntax, take the time to verify the result in Oracle documentation or by writing a small block of code. That way, your weighted score will decrease due to the increase in time, but you will at least get the question completely correct, which is more important than a fast time.
Certainly, if every quiz simply tested your ability to memorize syntax, the PL/SQL Challenge would become rather dull. I believe that I offer a nice blend of different kinds of questions and choices, which must be part of the reason that over 1200 developers, and usually more like 1400 developers, play the quiz each day.
Your thoughts?
SF
The PL/SQL Challenge (www.plsqlchallenge.com) offers a daily quiz on the PL/SQL language, through which thousands of Oracle technologists demonstrate and deepen their knowledge of PL/SQL. This blog contains posts by the PL/SQL Challenge founder, Steven Feuerstein, as well as comments from players.
1. I agree that the challenge should, from time to time, test our memorized knowledge of PL/SQL, because having an accurate knowledge of the details reduces how often we have to look things up in the documentation.
ReplyDelete2. The player's first objection here, however, is about the weighting being given to correct vs. incorrect answers, I think.
Normally, a question incorrectly marked as "yes" or "no" will only affect your score by 1/n (where n is the number of answers).
In this case, two of the possible answers to the question were the direct opposite of each other, which means if the player answers consistently, they will mark one as "yes" and the other as "no" - which means that being wrong affects your score by 2/5.
Personally, I don't think it's a big deal - however if the question had simply had "SYS_REFCURSOR" as one answer, or "SYSREFCURSOR" as one answer, instead of both, it would have been equally challenging, I think - and the weighting would have perhaps been a little less severe for people who get it wrong.
For me, using the documentation WHILE answering the question doesn't feel quite right. I will look something up immediately afterwards though (and maybe test an edge case). I suppose I'm treating the challenge like an interview or meeting.
ReplyDeleteSo I'll happily accept that I got lower marks for, by example, the result cache question. I know I don't really know that. Anyway, it is only one question out of about sixty a quarter.
But for people who do want to test code, it is worth knowing that you can do SQLs on apex.oracle.com which is an 11g EE environment. I have this vision of an April Fool's Day question starting "If create an empty table called DUAL..." and a thousand production databases suddenly going Kapow!
Steven is correct.
ReplyDeleteAs a programmer, we should know the syntax of the language.
It will be time consuming, to search oracle document or google for correct syntax.
This was the first time that you don't know what is the correct syntax for sys_refcursor
and you had lost some points in plsqlchallenge,
but you had also won the prize that
in future you will not confuse about the correct syntax of sys_refcursor.
:):):)
Sohil
Then the question arises - if copying of the code from the quiz page is considered as cheating then why typing-in of (presumably the same) block of code is not?
ReplyDeleteI was in the same jeopardy and with neither documentation nor a db at hand. I was banging my head against the wall for a minute or so (hence the slow response time) and rather than risking the double penalty opted to leave both choices blank.
ReplyDeleteThe important lesson is.... I knew of SYS_REFCURSOR, but not what it is good for.
Will this new knowledge revolutionalize my work? Surely not, but the PL/SQL Challenge has once again demonstrated its ability to make me a tiny bit smarter and I think that is what the quizzes are ultimately all about.
Regards
Mike
Hello Steven,
ReplyDeleteActually, I thought that the initial idea of PL/SQL challenge was to test real knowledge of PL/SQL without cheating like reading through the documenantion. I can get the necessary info from Pl/SQL doc very quickly. In case of SYS_REFCURSOR it was a matter of 5 sec. But again the assumption was not to read docs during the test.
Have you changed your mind letting us use the documenation?
The question was about WEAK cursors and not about the UNDERSCORE in SYS_REFCURSOR. It's one point.
Another one - it's not fair to charge twice for one "mistake" (I beleive it was not mistake in a sense of question). These two answer options were interdependent. Selecting one option you will never select another one.
To avoid such questions that was better to provide just one option related to SYS_REFCURSOR. That will be clear, because you will have to think just about if SYS_REFCURSOR is weak or not and not think about underscore or something like this.
But again I beleive that options were not correct.
Regards,
Sergey Porokh
The distinction between the SYSREFCURSOR and SYS_REFCURSOR choices is one of naming, not grammar or syntax. Assuming that a type named SYSREFCURSOR has been declared, the syntax is correct. Only one of the five choices presented was incorrect due to syntax ("TYPE REF CURSOR IS cursor_t").
ReplyDeleteAs PL/SQL software developers we should know the syntax of the language and at least be familiar with the facilities provided by STANDARD and DBMS_STANDARD. When compared to SQL, PL/SQL has a relatively small grammar that isn’t particularly difficult to learn. The real challenge is learning the details of the Oracle supplied types and packages and how to effectively use them within PL/SQL applications. Java is analogous in that its syntax may be taught in an afternoon but learning the myriad of classes requires extensive study. Perhaps it would be more apt to describe the PL/SQL Challenge as a test of one’s knowledge of the Oracle PL/SQL environment rather than the language.
I, too, have been adversely affected by questions that presented mutually exclusive choices. I’ve not been happy with my scores on those occasions, but they were due to my mistakes. Adjusting the scoring algorithm to "lessen the sting" in such cases would add complexity (and potential confusion) without significantly altering the quarterly results. Changing the scoring method could also result in cases where players are more harshly penalized for incorrect answers. The remedy others have suggested to avoid "double penalties" by treating mutually exclusive choices as single selections would logically require penalizing the player in full if any component of the group is incorrect. This could be particularly costly to a player who makes a mistake on a question where all of the choices are mutually exclusive.
Here's a comment from a player that I received via email (I will reply soon; I've been traveling):
ReplyDeleteThanks for keeping things going with the quiz it is adding to the knowledge of everyone who takes part and now the but ... but please take serious note of what correspondents like Sergey have to say. I also thought it was a very poor question 02/08 penalising people for what could well I've been a typo on the quizmaster's part (and there have been a few in other quizzes) or a trick question (i thought we weren't going to have trick questions?). If I and others want to promote this further so that the 1200 become 12000 and beyond (just imagine the correspondence) then less questions are needed where we have to look for typos; missing semicolons or /'s etc. It will just make people think "well I can't be bothered if there's stuff like that in there" which a decent compiler would pick up straight away. Still I've learnt today that answer didn't have an underscore in SYSREFCURSOR (my eagle eyes let me down) when I thought it was a question about weak cursors. In a funny way I'll probably not forget that now because of the oddity of this particular question. Looking out for the next missing semi-colon. (only joking - I learn/revisit something good in the quiz everyday). C'est la vie.
Had the SYSREFCURSOR option been presented without the SYS_REFCURSOR choice I might have assumed a typo had occurred; however, with both of them present it was clear to me that the intent of those choices was to demonstrate knowledge of the Oracle supplied type for declaring weak ref cursors. I didn't notice the difference in those two choices at first glance; I initially thought a choice had been repeated.
ReplyDeleteGenerally I assume that minor syntax or spelling errors are unintentional and answer accordingly. While I correctly surmised that the difference in these two choices was intentional, I can also understand how others might have come to a different conclusion.
1. When did Steven ever say that typing code, or looking up the documentation, constitutes cheating?
ReplyDeleteAccording to http://www.plsqlchallenge.com/pls/apex/f?p=10000:33:306332501143718::NO:::#cheating
it only says:
"You copy the contents of the quiz page into the clipboard, paste it into your PL/SQL environment and run the code."
So the rules explicitly disallow using the *clipboard*, but they don't disallow any other means of copying (e.g. typing).
(Aside: Plus, now that the questions are rendered as images, so even someone who wants to cheat via copy+paste will now find it a little harder; i.e. you'd have to copy the image into some OCR software first.)
Personally, I have no qualms about occasionally running a simple test case or doing a quick check in the documentation - it slows me down a bit, but it yields more accurate answers.
2. If there is a "minor syntax error", the answer is wrong, simple as that - otherwise you'd have to second-guess whether the questioner deliberately included the error or not.
Being able to detect minor syntax errors is a valuable skill; have you ever tried to unravel a compilation error that points to line #39, which was actually caused by a missing semicolon on line #10?
Jeff:
ReplyDeleteI completely agree with you on your first point. I view one’s score in the Challenge as a measure of one’s proficiency with PL/SQL; therefore, I don’t see any problem with referencing the documentation or typing in some code to run a test case. Taking the time to perform these activities lowers one’s score. Encyclopedic knowledge of the documentation may help one pass a certification exam, but it does not guarantee that one has the ability to apply that knowledge. Proficiency with PL/SQL is about much more than the ability to recall documentation details. Despite years of experience, I still frequently refer to Oracle’s documentation, especially when working with packages that I rarely use.
I also agree with your second point, at least in principle: any error, no matter how small, in an otherwise correct choice makes that choice incorrect. In practice, however, I recognize that even program code that has been validated may be inadvertently altered as it is incorporated into questions or choices. Players who reject a choice because of an unintentional syntax or spelling error should not be penalized – those players are, after all, correct. Players who fail to notice these same errors and accept a defective choice based on the broader concept being tested should also not be penalized. Steven’s approach to addressing such issues seams to be entirely adequate.
The ability to identify minor syntax errors is a valuable skill; however, it is a skill that is often only applied after the compiler has reported errors. Humans make extremely poor compilers – we often see what we expect even when it isn’t present. Knowing there is a code defect makes it much easier to find, even when it occurs somewhere other than where the error message indicates.
Ideally none of the quizzes will have unintended code defects; however, even when they are present I certainly wouldn’t want to be scored on my ability to be a compiler.
"And, clearly, if you want to make sure not to lose points for a dull issue of correct syntax, take the time to verify the result in Oracle documentation or by writing a small block of code. That way, your weighted score will decrease due to the increase in time, but you will at least get the question completely correct, which is more important than a fast time."
ReplyDeleteI did this exactly as I was not sure of the options :)
I appreciated Steven's approach too - where an unintentional mistake caused people to incorrectly mark an answer as right or wrong, he'll often correct the answer, and change everyone's answers and re-score accordingly, giving the benefit of the doubt to everyone who took the quiz.
ReplyDeleteOn another point, one good thing I learned at university was from a crotchety old professor who insisted that we learn to "compile code in our heads" before we let the computer do the work for us - a valuable discipline.
Yes, humans are poor compilers - but so are computers - they're faster and more consistent, but inferior when it comes to working out what's wrong and how to fix it :)
Hi there,
ReplyDeleteTalking here about cheating I used the followig source published on 27th April in News section:
"A player recently asked this question: "I read the news about mistakes identified by players on recent quizzes. It seems to me that if players relay on their own knowledge and not reading the Oracle documentation during the quiz, they would not have found these mistakes. Should players be relying on documentation to answer their questions?"
This is how we see things:
It would be great if everyone answered the quiz without looking at documentation or other resources. This would reflect their current knowledge of PL/SQL most accurately. Having said that, there is no way to stop players from consulting Oracle documentation or, for that matter, the contents of Steven's books."
...In addition to my point about wrong (i.e. weak) options for this "weak cursors" question:
If I say that SYS_REFCURSOR/SYSREFCURSOR is not a WEAK cursor type, i.e. I don't choose neither SYS_REFCURSOR nor SYSREFCURSOR I will get 20% points (or smth like that) more than choosing SYSREFCURSOR despite of the fact that the latter option is more likely to be true in terms of question and general sense.
Regards,
Sergey Porokh
Sergey, that FAQ content is changing to make it clear that looking in Oracle doc - or any doc - is perfectly fine.
ReplyDeleteSF
Often the documentation is an inadequate resource for guaranteeing a correct response. I got caught by this on the August 4 quiz. The option with "USING NULL" would seem to be correct by Oracle’s documentation which states that IN values may be either variables or expressions but offers no warning that NULL will raise an exception. Anytime I see NULL used as a value (which it is not) it gets more attention. In this case I consulted the documentation since the use of NULL in this case seemed suspicious. Because the documentation did not explicitly exclude NULL I changed my answer which unfortunately made it incorrect. I definitely should have tried a test case. As a software manual I encountered in the 80’s said, "In cases where the manual disagrees with the program, assume the program is correct."
ReplyDeleteJeff, I’m glad you had an instructor that encouraged you to evaluate your code before giving it to the compiler. Too many software developers have never acquired that skill and instead rely on the convenience of modern software development environments to enable them to hack away at their programs until they seem to work. It reminds me of a cartoon I saw where a programmer is staring at his screen. A coworker notices him and asks, "Trying to find the problem?” The programmer replies, "No, no. It works. I just don’t understand why!"
When I first developed an interest in computers I didn’t have access to one (dating myself here, but Apple wasn’t even a company yet). Using books from the library, I taught myself COBOL, FORTRAN, and PASCAL. The learning skills I developed through those endeavors have served me well, especially when exploring new software technologies.
"No, no. It works. I just don’t understand why!"
ReplyDeleteI love that. I reckon that's a mark of an excellent programmer - they are unsatisfied until they know why something works the way it does.
@jhall62: in the manual "PL/SQL User's Guide and Reference" exists a short chapter titled "Passing Nulls to Dynamic SQL" that explicitly describes that you can't pass the litaral value NULL for the USING clause (page 7-10 for 10g R2)
ReplyDeleteThanks, Niels. This is one of those cases where the Oracle documentation's organization can cause problems when only given a cursory viewing, which is exactly what is likely to happen when attempting to find an answer quickly for the quiz. For expediency I went straight to the syntax diagram for "EXECUTE IMMEDIATE..." and the description of "bind_argument" which is "An expression whose value is passed to the dynamic SQL statement, or a variable that stores a value returned by the dynamic SQL statement." Unlike the entry for "BULK COLLECT" which comes immediately after, the "bind_argument" doesn't even offer a link to more information.
ReplyDeleteJeff, I hadn't even thought of the cartoon from that perspective. My coworkers and I all viewed it as a case of accidental success instead of deliberate design (a most unreliable way to create quality software).