Two Years of the PL/SQL Challenge, and Looking Ahead

The second anniversary of the PL/SQL Challenge's launch date (8 April) approaches. Two years of daily quizzes, and so much more. Excellent time to reflect on how things have gone and how they can be improved in the future.

Three-plus years ago, I had this wild idea: that PL/SQL developers might like to take quizzes – even daily quizzes, both to learn more about PL/SQL and also to demonstrate to the world (well, mostly to other PL/SQL developers) how much of an expert they are in this database programming language.

Turns out I was right, since two years after starting the PL/SQL Challenge, thousands of developers have played these quizzes, submitting over 450,000 answers to hundreds of quizzes. Along the way, I learned many lessons about what it takes to write high-quality, error-free, interesting and non-trivial quizzes, and to provide a website where players could compete and be ranked. 

I offer below some thoughts on the PL/SQL Challenge and also alternative ideas for competitive play (the daily PL/SQL quiz) and international rankings. I would very much like to hear your own ideas, as well as any feedback you have on mine.

First, What's Coming
We will soon be releasing version 2.2 of the PL/SQL Challenge, with these major new features:

Activity Points

Up through 2.1, the only reflection of your activity on the PL/SQL Challenge was your ranking in the daily PL/SQL quiz, or some other quiz. Of course, the vast majority of players will never find themselves in the top rankings for these quizzes, but they nevertheless spend lots of time and effort at the PL/SQL Challenge improving their skills. In 2.2, we now award points to players for all of their activities on the site (taking quizzes, submitting quizzes, reviewing results, engaging in discussions about quizzes, etc.).

Practice Quizzes

Prior to 2.2, you could take a quiz when it is initially offered, and then review results and answers through the Library afterwards. You could not, however, take past quizzes if you'd missed them, no could you re-take a quiz. Now, you can set up practices based on past quizzes, specific features about which you want to learn more, or your favorite authors. You can also use the unique Autotune feature: the PL/SQL Challenge will automatically create practices for you, based on pre-defined criteria.

Quizbook: Quiz as PDF

One of the most long-sought enhancement requests on the PL/SQL Challenge, the Quizbook feature makes it easy for you to generate a PDF document of one or more quizzes. These can be formatted as an offline test (no answer information included), a player scorecard (showcasing your performance) or a knowledge document, containing all answers and resources. You can save and reuse your Quizbooks, as well as sharing them with other players.

On-Site Discussions

Rather than discuss interesting aspects of quizzes on the PLSQL-Challenge.Blogspot.com blog, you can now start or answer a thread directly on the Quiz Details page. You can also ask for help (it must be related to the topic of the quiz) or even post an objection to the quiz. In other words, if you think there is something wrong with the quiz, you can register your concern on both the post-quiz feedback page or on the Quiz Details page. You can also easily see if another player has already submitted a request for a correction.

Lessons Learned

Here are some of those lessons:

• You can't write a good quiz in five minutes. Maybe 30 minutes, when you include the time it takes to write verification code, test that code, etc. More like an hour. So, as you might well imagine, I have spent many hours over the past several years writing quizzes (and lots of other developers have, too, since over 200 quizzes have been submitted by others!).
• Many developers are ready to take the time and make the effort to play each day, but many more probably don't have the time or commitment or discipline to do any more than play occasionally, and use the website to look over quizzes and learn lessons from them.
• Moving from passive learning (reading documentation or books) to active learning through competition helps identify all sorts of nuances, bugs and areas for improvement in the technology covered by the quizzes.
• It is extremely difficult to come up with a way to offer quizzes that makes cheating impossible, but also extremely difficult to prove that anyone actually is cheating.
• It's hard work to support and maintain a website that is available 24x7. Oh yeah.
• A formal education in music theory is an excellent foundation for people who would like to become programmers. I have been very impressed and proud of how my son, Eli, has been able to come up to speed on SQL, PL/SQL, APEX, JavaScript, CSS and more.
• There's so much about the Oracle technology stack and running websites that I do not know. The PL/SQL Challenge could never have survived as long as it has without the help of my friends at Apex Evangelists.
• I sometimes wonder how long I can keep up the flow of five new quizzes each week that are of sufficient quality to be used as a way to rank PL/SQL developers.

Certainly, I could just keep the PL/SQL Challenge going the way it has, but I think that it is worth, after two years of publishing daily quizzes (and more), to contemplate alternatives to the daily quiz and, more broadly, public ranking through competition.

Variations on the Daily Quiz

I offer below ideas for variations on the daily PL/SQL quiz.The motivation behind these variations is that I want (a) lots more people to play quizzes generally and (b) lots more people to compete.

• Three new quizzes a week: rather than offer a new quiz each weekday, cut back on the volume by publishing a new quiz on, say, Monday, Wednesday and Friday. I will have to provide 40% fewer quizzes and players will not have to commit as much of their time to compete. The big question for players is: how important do you think it is to have a new quiz each day?
• Keep doing a daily quiz, but offer a mix each week of three new quizzes and two "used" quizzes – played sometime in the past (at least six months, probably, so that they will not be fresh in anyone's mind).
• Rather than post a new quiz each day, publish all the new quizzes (whether it be three or five) on, say, Sunday. They must be completed by the following Saturday, but you can choose when to take the quizzes. You might have a few hours available on Tuesday afternoon, so you do all the quizzes in one sitting.

Making Rankings Count

Here are some ideas to minimize the chance of cheating and improve the credibility of rankings:

• Rather than offer rankings on all quizzes taken, define only certain quizzes as competitive. Each player makes a decision to compete and to do so, you must register with a credit card, pay a small fee. If a person wants to compete under two accounts, they will have to go to much greater lengths to hide their identity (and they'll also have to pay more money). Sure it is still possible, but much less likely, I believe.
• For all other quizzes, you take them, you accumulate points for the effort, but you are not ranked. One advantage of making this distinction is that we can publish many more quizzes that will help you deepen your expertise (and highlight the knowledge of others), since not all quizzes will have to meet the criteria needed for competition. For example, "quick" true/false quizzes don't work well for the daily quiz, but can be very handy in reinforcing knowledge or exposing gaps.
• Your real name (that provided on your credit card, which must match your registration information) will be displayed in rankings. You can't "hide" behind a player name.
• Zero tolerance for aberrant scoring: everyone takes qualifier quizzes to verify their performance during the previous month (and, likely, for the quarterly championship as well). Highly aberrant patterns (such as extremely fast and extremely accurate) that cannot be reproduced lead to lifetime expulsion from the competition.'

Your Thoughts?

• What have you learned from the PL/SQL Challenge? 
• What do you like best about it? 
• How do you think it can be improved? 
• How can we get thousands of Oracle technologists to play? 
• Why don't your co-workers play now? 
• What can be done to get your manager to see the value of the PL/SQL Challenge, and actively encourage all members of the team to participate?

Many thanks in advance,
Steven Feuerstein