Q1 2011 Ends, Time for Next Playoff

Friends,

Q1 2011 ended on Friday, as far as the daily quizzes are concerned. Which means it is time to prepare for the championship playoff for that quarter. And a congratulations to the hundreds of players who committed a substantial amount of time playing quizzes, learning about PL/SQL, SQL and APEX, and in many cases sharing their knowledge with others through our discussion.

As noted in an earlier post, we are going to change the process for playoff qualification starting with this quarter: some or all of the top-25 ranked players will be required to take special qualifier quizzes to validate their performance in the quarter.

Here's how the qualifier process will work:

1. If you ranked in the top 25 and are not selected to take the qualifier quizzes, then you are in the playoff, with no further action required. Even if you cannot or choose not to participate in the playoff, there is no impact on your rankings or competitive status.

2. If you are selected to play qualifier quizzes, you will receive an email (and a message on the PL/SQL Challenge site) notifying you of the quizzes you must take. They will also, of course, appear on your PL/SQL Challenge home page. You will have three days in which to play the quizzes. If you cannot take the quizzes in that period of time, submit feedback to this effect and we will re-scheduled the quizzes.

3. If you do not want to take the qualifier quizzes, then change your status to non-competitive (visit your Account page, Preferences, and check the box next to "Play for education purposes only?") and notify us of your decision through the Feedback page. If you take this action, any awards you won in the quarter will be reversed.

4. After taking the qualifier quizzes, you will be notified of the results. If you pass, then you have qualified for the playoff and no further action is needed.

If you fail the qualifier, your status will be set to non-competitive and any awards received in Q1 2011 will be reversed. You will remain in non-competitive status for the remainder of this next quarter.

If you previously had your scores adjusted by our "very fast answer" analysis, your account may also be terminated.

For each qualifier failure, rankings will be re-calculated, thereby adding more players to the playoff. Those added to the top 25 may then be asked to take qualifier quizzes (and so the cycle above begins again).

Clearly, this process will delay the date on which the playoff will occur. In addition, the website must be upgraded to 2.2, with its support for qualifiers. We plan to do this upgrade the weekend of 13 April. So during the week of 15 April we will invite some players to take the qualifier quizzes.

Regards,
Steven Feuerstein

It's AMIS Month for the SQL Quiz!

I am very pleased to announce that the SQL quizzes for the month of April are provided by Lucas Jellema and Patrick Barel of AMIS (you will find at the end of this post a description of AMIS). Patrick is a longtime player at the PL/SQL Challenge, and also both reviews and authors SQL quizzes. Lucas is Chief Technology Officer of AMIS and a well-known and highly-regarded presenter at numerous Oracle conferences.

I hope that you enjoy their SQL quizzes. My deepest thanks to AMIS for supporting the Oracle community through the PL/SQL Challenge site.

Steven Feuerstein

About AMIS

AMIS is an Oracle, Java and SOA Specialist from The Netherlands, founded in 1991. Consistently active as frontrunner with the latest Oracle technology, AMIS is well known around the world for its community contributions, through presentations on conferences, Master Classes, articles in developer magazines, books and its Technology Blog (http://technology.amis.nl). AMIS has 5 Oracle ACEs and ACE Directors among its staff.

AMIS provides consultancy services around Oracle technology, including Database Development (SQL, PL/SQL, APEX); User Interface development using Java/JEE,ADF and WebCenter; SOA & BPM using SOA Suite, BPM Suite and OSB; BI using ODI, OBI EE; Security, Performance, Scalability, Availability and Administration of run time infrastructures including Database, Middleware and VMs. These services are rendered locally in The Netherlands, but also to customers in other countries and continents. Note that the Oracle Database, SQL and PL/SQL have always been crucial in the activities of AMIS.

AMIS is probably best known for its advanced knowledge, experiences and specialists. Knowledge transfer in various forms - training, coaching, reviews, "education permanente" - is a key element of what the company does. AMIS for example can deliver masterclasses and trainings on site (or through webinars) and also offers self-study materials.

The latest initiative from AMIS is the 'further' training program, targeted at experienced Oracle professionals, who are somewhat stuck in traditional tools and technology and perhaps yesterday's versions or ways of working, and who are now ready and eager to 'further' their knowledge and career. This program is offered to individual developers as well as to organizations who have teams to propel in this fashion. The program is provided in various ways and through various channels.

More information about AMIS can be found at the corporate web site, the Twitter account - @AMIS_Services, its technology blog, and through email to info@amis.nl.

A Hierarchy of One? (11633)

The 28 March quiz on object type hierarchies asked players to identify those keywords that "that are needed to create an object type hierarchy". Several players wrote to object to the way the quiz was scored:

1. "The question for this quiz was: Which of the choices are keywords that are needed to create an object type hierarchy? (with the word "hierarchy" in bold letters). You have marked the answer "CREATE TYPE" ([ 10065 ]) as correct with explanation "We use the CREATE TYPE statement to create an object type, so this is definitely needed!". Of course, it is obvious that you cannot create a type without CREATE TYPE keywords, but with "hierarchy" being bolded in the question I concluded that we only need to mark the keywords that are strictly used for creating object type hierarchies. That's why I left this answer unmarked. I think you should mark this answer as correct for everyone because of ambiguity in the question."

2. "In my opinion the keyword "INSTANTIABLE" is needed for at least one class because otherwise you can't instantiate an instance of any object in your hierarchy."

3. " I do not agree with your evaluation of the right answers. You have NOT mentioned that the hierarchy is supposed to have AT LEAST two levels. It is true that a hierarchy with only one level is trivial, but it is still a hierarchy. For such trivial hierarchies the first two choices are not mandatory. This is like the empty set in mathematics. It is still a set even it has no element. It is trivial but exists!"

And my responses:

1. I don't think you can say "Of course" I am correct, but then believe that the choice is not correct. I understand your concern about the word hierarchy being bolded, but it doesn't take away from the fact that you need CREATE TYPE to accomplish this.

2. INSTANTIABLE is the default for an object type and method, so I don't see why this would be needed.

3. While your objection may be theoretically correct, I think that it is plenty clear from the emphasis on hierarchy that the intention of the question was to identify the keywords needed to create a non-trivial hierarchy.

I do not believe that any change in scoring is required. Your thoughts?

Cheers, Steven

Ah, Those Edge Cases! (11632)

The 27 March quiz tested your knowledge of collections, and the kinds of assumptions you might make when writing a program that accepts a collection as a parameter.

The way the quiz was formulated was different from the usual, and it definitely caused some frustration (one player wrote "Maybe it is because I was awake for about 2 hours last night doing some DBA work for a client, but I did not even understand the question that was being asked. Even after looking in the Library to see the Topic/Summary, my co-worker and myself are still confused on what exactly was being asked." and another: "This question was confusing. I assumed that the requirements were the requirements of the code that was being called. In other words, if all the conditions are met the code will work properly, not does this code enforce all the conditions.").

Our very smart and dedicated reviewers caught many issues with this quiz, and we spent a fair amount of time discussing ways to tighten up the quiz (and I followed almost all of their advice). Yet, in the end, we were "defeated" (that is, an issue was identified in the way the quiz was scored) by an edge case regarding string-indexed arrays - which was covered in the 22 March quiz.

Several players pointed out that choice 9991 (the choice that was scored as correct and stated that no requirements applied) would require an integer-indexed array, since the algorithm would not correctly display all elements in the collection if one of the index values was the empty string.

How right they are, as one player demonstrated with a script:

SQL: CREATE OR REPLACE PACKAGE plch_pkg AS
  2     TYPE my_collection IS TABLE OF VARCHAR2(10) INDEX BY VARCHAR2(20);
  3     index_type VARCHAR2(20);
  4  END plch_pkg;
  5  /
 
Package created

SQL: CREATE OR REPLACE PROCEDURE plch_proc(numbers_in IN plch_pkg.my_collection) IS
  2     l_index plch_pkg.index_type%TYPE := numbers_in.FIRST;
  3  BEGIN
  4     WHILE (l_index IS NOT NULL) LOOP
  5        DBMS_OUTPUT.put_line(numbers_in(l_index));
  6        l_index := numbers_in.NEXT(l_index);
  7     END LOOP;
  8  END;
  9  /
 
Procedure created

SQL: DECLARE
  2     l plch_pkg.my_collection;
  3  BEGIN
  4     l('b') := 'b';
  5     dbms_output.put_line('without null index');
  6     plch_proc(l);
  7     l('') := 'a';
  8     dbms_output.put_line('with null index');
  9     plch_proc(l);
 10  END;
 11  /
 
without null index
b
with null index
 
PL/SQL procedure successfully completed

Another players also wrote to say that "There is a requirement that the collection be non-empty, but no requirement that the collection is initialized (relevant for nested tables and varrays)." This is true. We did not include as a requirement in the list the need for a collection to be initialized. And we did not specify that you should only consider the requirements in the list.

What's a quiz administrator to do?

1. My biggest problem with this quiz is the possible complications for future quizzes. It is one (interesting) thing to know that an empty string as index value can cause odd behavior. It is quite another thing to have to account for this scenario in every quiz on string-indexed arrays. Seems like an unnecessary complication. So I plan to add to the assumptions for the daily PL/SQL quiz that unless otherwise mentioned, all index values of string-indexed collections have a non-NULL length. Any objections to my doing this?

2. Add to this quiz a statement to the effect that only those requirements listed in the question are to be considered. That is, there might be other requirements (such as "The nested table or varray must be initialized.") that could apply, but are not to be considered in this quiz.

3. Give credit for a correct answer on 9991 to all players who submit feedback indicating they marked this choice as incorrect because of the possibility of an empty string index value. I very much doubt that many players recalled this edge case and applied it to this quiz, but if you did, let us know and we will give you credit.

4. Give credit to any player who marked all choices as incorrect and submit feedback indicating that they did so because we did not include a requirement specifying initialization of nested tables and varrays. I say "all choices as incorrect" because if you thought of this additional requirement, then none of the choices could be correct (only the first even includes the "must be associative array" requirement).

Did I miss anything?

Two Years of the PL/SQL Challenge, and Looking Ahead

The second anniversary of the PL/SQL Challenge's launch date (8 April) approaches. Two years of daily quizzes, and so much more. Excellent time to reflect on how things have gone and how they can be improved in the future.

Three-plus years ago, I had this wild idea: that PL/SQL developers might like to take quizzes – even daily quizzes, both to learn more about PL/SQL and also to demonstrate to the world (well, mostly to other PL/SQL developers) how much of an expert they are in this database programming language.

Turns out I was right, since two years after starting the PL/SQL Challenge, thousands of developers have played these quizzes, submitting over 450,000 answers to hundreds of quizzes. Along the way, I learned many lessons about what it takes to write high-quality, error-free, interesting and non-trivial quizzes, and to provide a website where players could compete and be ranked. 

I offer below some thoughts on the PL/SQL Challenge and also alternative ideas for competitive play (the daily PL/SQL quiz) and international rankings. I would very much like to hear your own ideas, as well as any feedback you have on mine.

First, What's Coming
We will soon be releasing version 2.2 of the PL/SQL Challenge, with these major new features:

Activity Points

Up through 2.1, the only reflection of your activity on the PL/SQL Challenge was your ranking in the daily PL/SQL quiz, or some other quiz. Of course, the vast majority of players will never find themselves in the top rankings for these quizzes, but they nevertheless spend lots of time and effort at the PL/SQL Challenge improving their skills. In 2.2, we now award points to players for all of their activities on the site (taking quizzes, submitting quizzes, reviewing results, engaging in discussions about quizzes, etc.).

Practice Quizzes

Prior to 2.2, you could take a quiz when it is initially offered, and then review results and answers through the Library afterwards. You could not, however, take past quizzes if you'd missed them, no could you re-take a quiz. Now, you can set up practices based on past quizzes, specific features about which you want to learn more, or your favorite authors. You can also use the unique Autotune feature: the PL/SQL Challenge will automatically create practices for you, based on pre-defined criteria.

Quizbook: Quiz as PDF

One of the most long-sought enhancement requests on the PL/SQL Challenge, the Quizbook feature makes it easy for you to generate a PDF document of one or more quizzes. These can be formatted as an offline test (no answer information included), a player scorecard (showcasing your performance) or a knowledge document, containing all answers and resources. You can save and reuse your Quizbooks, as well as sharing them with other players.

On-Site Discussions

Rather than discuss interesting aspects of quizzes on the PLSQL-Challenge.Blogspot.com blog, you can now start or answer a thread directly on the Quiz Details page. You can also ask for help (it must be related to the topic of the quiz) or even post an objection to the quiz. In other words, if you think there is something wrong with the quiz, you can register your concern on both the post-quiz feedback page or on the Quiz Details page. You can also easily see if another player has already submitted a request for a correction.

Lessons Learned

Here are some of those lessons:

• You can't write a good quiz in five minutes. Maybe 30 minutes, when you include the time it takes to write verification code, test that code, etc. More like an hour. So, as you might well imagine, I have spent many hours over the past several years writing quizzes (and lots of other developers have, too, since over 200 quizzes have been submitted by others!).
• Many developers are ready to take the time and make the effort to play each day, but many more probably don't have the time or commitment or discipline to do any more than play occasionally, and use the website to look over quizzes and learn lessons from them.
• Moving from passive learning (reading documentation or books) to active learning through competition helps identify all sorts of nuances, bugs and areas for improvement in the technology covered by the quizzes.
• It is extremely difficult to come up with a way to offer quizzes that makes cheating impossible, but also extremely difficult to prove that anyone actually is cheating.
• It's hard work to support and maintain a website that is available 24x7. Oh yeah.
• A formal education in music theory is an excellent foundation for people who would like to become programmers. I have been very impressed and proud of how my son, Eli, has been able to come up to speed on SQL, PL/SQL, APEX, JavaScript, CSS and more.
• There's so much about the Oracle technology stack and running websites that I do not know. The PL/SQL Challenge could never have survived as long as it has without the help of my friends at Apex Evangelists.
• I sometimes wonder how long I can keep up the flow of five new quizzes each week that are of sufficient quality to be used as a way to rank PL/SQL developers.

Certainly, I could just keep the PL/SQL Challenge going the way it has, but I think that it is worth, after two years of publishing daily quizzes (and more), to contemplate alternatives to the daily quiz and, more broadly, public ranking through competition.

Variations on the Daily Quiz

I offer below ideas for variations on the daily PL/SQL quiz.The motivation behind these variations is that I want (a) lots more people to play quizzes generally and (b) lots more people to compete.

• Three new quizzes a week: rather than offer a new quiz each weekday, cut back on the volume by publishing a new quiz on, say, Monday, Wednesday and Friday. I will have to provide 40% fewer quizzes and players will not have to commit as much of their time to compete. The big question for players is: how important do you think it is to have a new quiz each day?
• Keep doing a daily quiz, but offer a mix each week of three new quizzes and two "used" quizzes – played sometime in the past (at least six months, probably, so that they will not be fresh in anyone's mind).
• Rather than post a new quiz each day, publish all the new quizzes (whether it be three or five) on, say, Sunday. They must be completed by the following Saturday, but you can choose when to take the quizzes. You might have a few hours available on Tuesday afternoon, so you do all the quizzes in one sitting.

Making Rankings Count

Here are some ideas to minimize the chance of cheating and improve the credibility of rankings:

• Rather than offer rankings on all quizzes taken, define only certain quizzes as competitive. Each player makes a decision to compete and to do so, you must register with a credit card, pay a small fee. If a person wants to compete under two accounts, they will have to go to much greater lengths to hide their identity (and they'll also have to pay more money). Sure it is still possible, but much less likely, I believe.
• For all other quizzes, you take them, you accumulate points for the effort, but you are not ranked. One advantage of making this distinction is that we can publish many more quizzes that will help you deepen your expertise (and highlight the knowledge of others), since not all quizzes will have to meet the criteria needed for competition. For example, "quick" true/false quizzes don't work well for the daily quiz, but can be very handy in reinforcing knowledge or exposing gaps.
• Your real name (that provided on your credit card, which must match your registration information) will be displayed in rankings. You can't "hide" behind a player name.
• Zero tolerance for aberrant scoring: everyone takes qualifier quizzes to verify their performance during the previous month (and, likely, for the quarterly championship as well). Highly aberrant patterns (such as extremely fast and extremely accurate) that cannot be reproduced lead to lifetime expulsion from the competition.'

Your Thoughts?

• What have you learned from the PL/SQL Challenge? 
• What do you like best about it? 
• How do you think it can be improved? 
• How can we get thousands of Oracle technologists to play? 
• Why don't your co-workers play now? 
• What can be done to get your manager to see the value of the PL/SQL Challenge, and actively encourage all members of the team to participate?

Many thanks in advance,
Steven Feuerstein

Over 450,000 Answers to 681 Quizzes!

Today, the PL/SQL Challenge hit another milestone: players have submitted over 450,000 answers to N quizzes since April 2010.

 We just passed a big milestone at the PL/SQL Challenge: over 450,000 answers have been submitted for the over 680 quizzes offered since its inception in April 2010. Here are some stats:

The Stats	Wow!
Number of Registrations	10,603
Number of Players	8,740
Quizzes Played	681
Answers Submitted	450,424
Time Spent Answering Quizzes	1,053 days
Average Time Per Quiz	03 mins 39 secs
Prize Money Awarded	$40,097

Many thanks to all the players who have:

• Taken the time to play consistently over the last two years;
• Submitted quizzes for play; 
• Reviewed quizzes to help improve their quality (tremendously!);
• Spread the word about the PL/SQL Challenge.

And thanks also to my friends at Apex Evangelists, who played a key role in getting the PL/SQL Challenge up and running, and who help us keep it going.


Steven Feuerstein

New Qualifier Process for Playoffs

We will be instituting a new process for the upcoming Q1 2012 championship playoff.

All or some of the players who qualify to participate in the playoff will be required to take three qualifier quizzes.

If the performance on these qualifier quizzes is substantially worse than the player's performance during the quarter (% correct and/or time required to answer the quiz), then the following steps will be taken:

1. That player will be ineligible to participate in the playoff.
2. That player's status will be set to non-competitive.
3. Any awards from the previous quarter will be reversed.

We believe that this qualifier step will strengthen the integrity of the playoff and ensure the best possible results for all players.

Warm regards,
Steven Feuerstein