18 July 2011

Tricky choice tricks the quiz author! (4720)

In the logic puzzle for the week of 9 July, we scored the following choice as correct:

If 1 is in the solution then the other numbers must be (2, 4, 7).

The explanation given was this: "If 1 is in the solution, then the third turn tells us that (3, 6, 5) cannot be in the solution. The only 3 numbers left are (2, 4, 7) and looking over the first two turns, they would be compatible. So the four numbers (1, 2, 4, 7) could be in the solution together."

The third turn showed that (3 1 6 5) had just one correct digit, not in the right location.

As two players pointed out to me, the third turn shows that (2 4 7) have to be in the solution, regardless of the status of 1. We need four digits, the third turn tells us that at most one of (1 3 6 5) are in the solution, so we need all of the other three digits (2 4 7).

Thus, 2, 4 and 7 must be in the solution - but that fact has nothing to do with whether or not 1 is in the solution.

In other words, even if 1 is not in the solution, 2, 4 and 7 must still be in the solution.

So what's a poor quiz administrator to do about this? I was at first tempted to do what I usually do: give everyone credit for a correct answer to this choice, and change the text to get rid of this "problem."

But then I thought: wait a minute, the whole point of this puzzler is to test our facility with logical thinking. Clearly, I have demonstrated that I am not as good at this sort of thing as I thought (and by the way, 62% of all players selected this choice as correct).

So it seems to me that what I should do is (a) change this choice to be incorrect; (b) recalculate the scores of all players; (c) change the difficulty level of this quiz to Advanced.

What do you think, dear players?

10 comments:

  1. Hello Steven,

    As one of those who raised this problem, I still think the following:

    Judging SOLELY by the boolean logic rules, if two statements A and B are both TRUE,
    than the implication "A implies B" is also TRUE.

    It (that is, the implication itself !) CANNOT be false by any means, because that would require at least one case where A is TRUE and B is FALSE.

    The only problem with this "IF" statement is that it is "weeker" than the assumptions allow it.

    It could have been replaced with 2 stronger statements, one saying that "1" is in the solution, and the other one saying that "2,4,7 are in the solution", both of them being true.

    The "IF" statement could have been stronger in case that we could only conclude from the clues that "ONLY" if 1 is in the solution, then 2,4,7 are also in the solution.

    Otherwise, it is a "week true" statement, but still NOT a false statement.

    So, in my opinion, you can BY NO MEANS score it as incorrect, that would be an undeserved punishment for those that not only realised it as being logically correct
    (by the truth table of the boolean implication operation), but also objected on the fact
    that it was not the "strongest" statement that can be derived from the clues.

    The quiz was already rated as Advanced, so I think you have nothing to change.

    In the worst case, you can credit everyone with a correct choice, though, you can never know
    what was the reason for which those that marked it as incorrect have made their choice this way.

    May they have a bonus because of the ambiguity, but those that made a strong and correct rationing ( even more correct than was stated in the answers themselves ) cannot be punished,
    that would be completely unfair.

    Thanks a lot & Best Regards,
    Iudith Mentzel

    ReplyDelete
  2. The choice "If 1 is in the solution then the other numbers must be (2, 4, 7)" is a correct choice. Had the predicate been phrased as "If, and only if, 1 is in the solution," then the choice would have been incorrect since the third turn requires that the numbers (2, 4, 7) be in the solution.

    Because the choice is correct, there is no reason to recalculate scores.

    Because the third turn is sufficient to determine three of the four digits (which greatly facilitates finding the solution), the quiz should not be classified as advanced.

    ReplyDelete
  3. I must agree with Iudith and jhall62. The choice is correct. No rescoring needed from my point of view.

    Kind regards,
    Marga

    ReplyDelete
  4. Why should the choice be re-scored? If 1 is in the solution then 2,4 and 7 are too. That is a true statement!

    Looking at the numbers in turn 3, one at a time, we rule out 3, 5 and 6. 1 is left as the only number that fits in all three turns. We derived that 2,4 and 7 must be in the solution. Isn't that pretty much what the choice says?

    Who says we must start out by making a rule from the obvious result of turn 3, telling us that 2,4,and 7 are in the solution no matter what? And what difference does it make?

    The explanation is consistent with the choice.

    Mike

    ReplyDelete
  5. If you take a look at a truth table for implies, then F => T is a true expression. So no rescoring needed.

    ReplyDelete
  6. I was the 2nd one who brought this to Steven's attention.

    You'd be all right, if we were talking about boolean expressions. But we don't, it's a real world sentence. And the question was about a "valid conclusion". In this case, the conclusion was wrong, that it depends on 1, so for me the answer was wrong itself (as in "If my phone rings today, milk is white").

    I nevertheless wouldn't rescore all the boolean guys... ;-)

    Roger

    ReplyDelete
  7. The fact that if your phone rings today, milk is indeed white, and worse, even it it doesn't ring, it is white, so thank you for proving our point, unintentionally. It is a True statement! Roger, this quiz is about logic. And in this game, statements like that are logic statements and could not be handled else. Language could be ambiguous, but not in this situation.

    What can be discussed is the usefulness of the specific statement, but not the truth value of it.

    ReplyDelete
  8. Hello All,

    This thread seems to become more and more "colorful" :) :)

    However,

    1. As Wim said above, F => T is true, and, in our case we have in fact T => T, which is also true.

    2. Roger probably felt the same as me about the IF statement, however, as JHall62 correctly underlined above, the IF statement would have been incorrect only if it were written as
    "IF and ONLY IF "1" is in the solution, then ...", which would have meant that,
    if 1 is NOT in the solution, then the second part might be false, which is not the case.

    Or, in your terms, if you would say "If and ONLY if the phone rings today, then milk is white",
    which is obviously false.

    To just make an analogy with a recent quiz from the SQL competition, if we have a WHERE condition like
    "col_a > ALL (select col_b from mytable)",
    then if the inner select returns no rows, the
    WHERE condition itself always TRUE,
    because there is no "sample case" that would contradict it.
    On the other hand, a similar condition with
    ANY instead of ALL is always false.

    Thanks & Best Regards,
    Iudith Mentzel

    ReplyDelete
  9. I'm with those who say to leave the choice as correct. The statement is true as written, since it said "if", not "if and only if".

    If the choice is false, then there must exist some answer which includes a 1, but in which the other numbers are not 2, 4, and 7. This cannot be true, since we know that the solution must include 2, 4, and 7, regardless of the value of the fourth number. Just because the second part of the condition is true regardless of the truth of the first part does not make the statement false.

    Of course, if there had been enough evidence to rule out the possibility of 1 being in the solution at all, then scoring would be trickier. I'd still hold that the statement is true, personally, but one could argue against it.

    ReplyDelete
  10. Thanks, all! I will certainly follow the overwhelming recommendation to not change the scoring. I will, however, strive to avoid such conditional statements in the choices, in the future, and stick instead to unconditional claims.

    ReplyDelete