Pop Quiz: Falsifiable Hypotheses

This is a short quiz for your course on Scientific Thinking.

Take time to think about your answer before peeking at the solution.

Have your cake and eat it too

From Robyn Dawes’s Rational Choice in an Uncertain World:

Post-hoc fitting of evidence to hypothesis was involved in a most grievous chapter in United States history: the internment of Japanese-Americans at the beginning of the Second World War. When California governor Earl Warren testified before a congressional hearing in San Francisco on February 21, 1942, a questioner pointed out that there had been no sabotage or any other type of espionage by the Japanese-Americans up to that time. Warren responded, “I take the view that this lack [of subversive activity] is the most ominous sign in our whole situation. It convinces me more than perhaps any other factor that the sabotage we are to get, the Fifth Column activities are to get, are timed just like Pearl Harbor was timed… I believe we are just being lulled into a false sense of security.”

So, Warren says that the Japanese-Americans were probably plotting to sabotage the US from inside. Somebody notes that they haven’t found any sabotage. Warren says that is an ominous sign - it convinces him even more.

What is the problem with Warren’s reasoning?

Solution


Answer: There is no outcome that will falsify his belief. Take the hypothesis that a Fifth Column exists. Then, if you expect sabotage to be likely, then you must necessarily expect lack of sabotage to be unlikely - they are mutually exclusive. If you believe that a Fifth Column exists and you see no sabotage, that should decrease your confidence in that hypothesis. Instead, he says that it increases his confidence, as though he had predicted it all along. Wrong. He has an unfalsifiable belief and is using that to push a policy he wants.

How will you falsify these beliefs?

Your friend says that he believes in astrology. How can you falsify that theory?

Solution


One way could be to take someone whose horoscope predicts a good day. Then, make it bad - punch him in the face, throw eggs at him, whatever. The opposite is, of course, to take someone who’s supposed to be having a bad day, and then give them $1000 or whatever.

How can you falsify Newton’s Second Law of Motion?

Solution


If you apply a net force of F to an object of mass m, the Law forbids that the object will have an acceleration of anything but F/m (ignoring experimental error).

A scientist comes to you and says that smoking causes cancer. What would falsify his theory? What about all the smokers who haven’t got cancer? Does that falsify his theory?

Solution


If no chain-smokers ever got any lung cancer, that would be a severe blow to the theory.
Next, note that not all smokers have to get cancer - there are a lot of factors that determine how likely you are to get cancer. As such, “smoke and you might get cancer” is a vague but falsifiable hypothesis - not very precise, but still forbids that no smokers get cancer.

Person A: “No democracy will start a war.”

Person B: “But Iraq, Serbia, Bosnia, etc. have all launched wars.”

Person A: “Ah yes, but no true democracy will start a war.”

What’s wrong with A’s theory?

Solution


It is unfalsifiable. He will define all democracies who start wars as not true democracies.

A certain self-help author, who shall remain unnamed, says that if you follow the techniques in his book ($12.41 from Amazon), you will be able to build a lifestyle where you have to work only four hours per week. How can you possibly falsify his hypothesis? What do such authors do to protect themselves from being falsified?

Solution


If you followed his techniques and still didn’t get a four-hour workweek, then his “theory” is wrong. So, he will make excuses saying that you didn’t actually follow all the techniques properly.

Consider this fable:

If a tree falls in a forest and no one hears it, does it make a sound? One says, “Yes it does, for it makes vibrations in the air.” Another says, “No it does not, for there is no auditory processing in any brain.”

What is wrong about their argument?

Solution


They’re arguing about words. They have no differences in prediction. Think: do they expect different things from a vibration-detector or brain scan? Do they expect a sound recorder to play differently?

You get pimples on your face one day. Your friend says that it’s because you’ve been very stressed lately. That’s a good explanation, right? You have been stressed lately. What’s wrong with the theory, then?

Solution


What about all the times you had stress and didn’t get pimples? Since you don’t generally notice the lack of pimples, you don’t remember what his theory predicts and thus don’t realize that it is falsified. This way, his theory is safe from being proved wrong.

Notes

Most of the questions are from examples in Eliezer’s sequence on Mysterious Answers to Mysterious Questions.

The CSS for the spoiler solutions is thanks to this StackOverflow question.