Loading
Here is a short question. What would make you drop a belief you hold? Most people answer fast. They say “good evidence.” They say “real proof.” Neither answer names a single thing. A real answer names something that could actually happen to you.
That is a harder question than it sounds. It is also the only honest test of an open mind. Not “have you heard the other side.” This one.
You met the word falsify in the Logic and argument module. You met claims that no test could ever check. This module is not about the word. It is about the habit. The habit is naming what would beat you, before it shows up.
Nobody does this by accident. It has to be said out loud, or written down. The rest of this module is what that looks like.
Start with a small one. Two people believe the same thing about the same tea. One of them has done something the other has not.
Which of these beliefs is in better shape?
In 1919, a young man named Karl Popper met a famous doctor. The doctor was Alfred Adler. He had his own theory of people. Popper helped in Adler's clinics for children in poorer Vienna. Popper brought him a case that did not fit the theory. Adler explained it in his theory anyway, without meeting the child. Popper asked how he could be so sure.
Adler answered that his certainty came from a thousand past cases. Popper heard himself reply before he could stop.
“And with this new case, I suppose, your experience has become thousand-and-one-fold.”— Karl Popper, “Science: Conjectures and Refutations,” in Conjectures and Refutations (1963)
It was a joke. It was also the whole problem, in one line.
Later, Popper worked out what had bothered him. The theory did not just fit that child. It fit everybody. A man who gives his money away fits it. A man who hoards money fits it too. Whatever a person did, the theory had a reason ready and waiting.
“I could not think of any human behaviour which could not be interpreted in terms of either theory.”— Karl Popper, Conjectures and Refutations (1963)
This is the part that catches careful people. A theory like that never fails in front of you. It looks unbeatable.
This is hard to spot because the wins keep arriving. Every new case fits, and confidence piles up. But a theory that fits everything never told you what to expect. So counting its wins tells you nothing.
That winter, Popper turned the whole thing into a rule. Years later he wrote it out plainly, and it still surprises people.
“A theory which is not refutable by any conceivable event is nonscientific. Irrefutability is not a virtue of a theory (as people often think) but a vice.”— Karl Popper, “Science: Conjectures and Refutations,” in Conjectures and Refutations (1963)
“Every ‘good’ scientific theory is a prohibition: it forbids certain things to happen. The more a theory forbids, the better it is.”— Karl Popper, Conjectures and Refutations (1963)
So the question is not what your belief explains. The question is what your belief rules out. If it rules out nothing, then nothing can ever go against it.
Try it on four everyday beliefs. Ask what each one forbids.
Which of these beliefs rules something out?
Tap an item, then a bucket
A belief that can lose usually has three parts. It names a thing that could happen. It names how big. It names by when. “My knee will ache” has none of them. “My knee will ache on most rainy days this month” has all three.
You do not need to be exact. You need to be beatable. A rough number with a date beats a careful word with none.
In March 1951, a United States intelligence office wrote a short report. It said an attack on Yugoslavia that year was a serious possibility. The report went to people who had to plan around it. A few days later, one of them asked a simple question. What odds did you have in mind?
One of the people behind the report was Sherman Kent. He went back and asked his own colleagues the same question. They had all signed off on that one phrase.
What odds had his colleagues meant by “serious possibility”?
“It was another jolt to find that each Board member had had somewhat different odds in mind and the low man was thinking of about 20 to 80, the high of 80 to 20.”— Sherman Kent, “Words of Estimative Probability,” Studies in Intelligence (CIA, 1964)
One man meant it was unlikely. Another meant it was likely. The phrase covered both, so no event could ever prove it wrong. Vague words are comfortable for exactly the reason they are useless.
The fix is not a bigger vocabulary. It is a number and a date.
Tap to sharpen the same belief.
Same opinion, same person. The second names a thing, a size and a date. So it can fail.
Popper had one example he kept returning to. Einstein's theory said that heavy things pull on light. So stars near the sun should look shifted slightly away from it. You cannot see those stars in daylight. During an eclipse you can photograph them. Compare that photo with a night photo, and you have your answer.
“Now the impressive thing about this case is the risk involved in a prediction of this kind. If observation shows that the predicted effect is definitely absent, then the theory is simply refuted.”— Karl Popper, Conjectures and Refutations (1963)
The photographs were taken in 1919, and the shift was there. But being right is not the interesting part of this story. The interesting part is that one afternoon could have ended the theory.
Here are the moves, in the order that keeps you honest.
Put these steps in order. They turn a vague belief into one that could lose.
Put these in order
There is a second way to make a belief safe. It looks careful. You set a bar that no evidence could ever clear. Say a study of a million people, run for fifty years. That sounds strict. It is the same protection, wearing a lab coat.
This one is worth sitting with, because it wears good clothes.
This is hard to spot because it sounds like care. But a bar no real study could clear does nothing. The belief is safe either way, and nobody sees it happen. A fair condition uses evidence people can really collect.
Three common complaints, each with a version that could fail.
Match each vague belief to the version that could lose.
Tap one side, then its match
New facts arrive. People tend to do one of two things with them. Some never move at all. The belief was true before, so it stays true. Others flip the whole way. One story, and the old belief is gone. Both look like an answer. Neither one is careful.
Never moving keeps you safe from being wrong today. Flipping makes you feel fair and open-minded. Both of them throw away the size of the evidence.
There is a third option, and it feels odd at first. You hold a belief at a strength. Not a yes or a no. Say you are 7 in 10 sure the bus route is worse. A fact arrives. You move to 6 in 10, or up to 8. You have not flipped. You have not frozen. You have moved.
This is not a trick for maths people. It is a way of speaking. Anyone can say a number out of ten.
You might think the exact number is fussy. Surely the word is enough. Researchers checked that directly.
They studied 888,328 real forecasts about world events. They rounded the exact numbers into words like “likely.” What happened to accuracy?
Another team looked at how forecasters changed their minds over time. They had over 400,000 forecasts on almost 500 questions. The most accurate people updated often, in small steps. The least accurate sat still, or leapt a long way at once.
How far should one fact move you? There is a plain question for that. If I were wrong, would I still expect to see this? If yes, the fact is weak. It fits both stories, so it settles nothing. If no, the fact is strong. It is hard to explain unless you were right.
A friend says her charm brings good weather. Today is sunny. Would you expect sun anyway? Yes, plenty of days are sunny. So the sunshine is weak. It fits the charm and it fits no charm.
Run that question on four facts about the same belief.
You believe the new bus route is slower. Which facts push hard?
Tap an item, then a bucket
People hear a number out of ten and think you are dodging. It is the opposite.
It feels that way because school taught us to defend one answer. But 7 in 10 is sharper than “probably,” not softer. It says exactly how wrong you are willing to be. Someone can score it later. “Probably” never can.
Here is the whole habit in one small, annoying situation.
Your tap drips. You think the landlord will never fix it. What do you do with that belief?
You have told the landlord twice, and nothing happened. You are about 8 in 10 sure nothing ever will.
All of this only works if you write the line down first. Once the result arrives, the line is very hard to find again. You met that effect in the Cognitive biases module. The past looks obvious. The written note is not a rule for yourself. It is a witness.
The next few sections are about people who tried it at scale. They are scientists, which makes the results more uncomfortable, not less.
Some journals now let scientists lock in a prediction before collecting data. The journal accepts the paper on the plan alone. Any answer gets printed. Researchers compared 152 normal papers with 71 locked-in ones.
In how many did the result back up the researchers' first prediction?
That comparison is worth knowing where it came from. The team looked at the first prediction made in each paper.
The same pattern turns up in medicine. Around the year 2000, large trials started being registered in advance. Two researchers went through 55 big heart trials from 1970 onward. Before 2000, 17 of 30 trials reported that the treatment worked. After 2000, 2 of 25 did.
Writing it down first only helps if the note is kept. One team checked 67 trials in five leading medical journals. They compared each published paper against what it had promised to measure. In 58 of them, the published results did not match the promise. Results went missing, and new ones appeared that were never planned.
You do not need a journal to do this. There is a sentence for it.
This is hard to spot because it is usually sincere. The person is not lying about the past. They are reading it off a memory that has already tidied itself. Nothing in the sentence can be checked. Nobody can push back, including them. A dated note is the only thing that ever settles it.
The tool itself is small. Four lines, and it fits on a napkin.
Tap through the four lines.
One sentence, in your own plain words. “The new route makes me late.”
On the day, you read the note. One of three things is true. The thing you named happened, so you move a long way. It did not happen, so you move the other way. Or you find yourself explaining why the line does not count. That third one is the interesting case.
In 1975, researchers ran a study on how beliefs stick. Students did a task, then heard how well they had done. Some heard they were very good at it. Some heard they were bad. The scores were fake. They had been decided before the task began. Then the researchers explained all of that, plainly, to everyone.
The only reason for the belief was gone. Everyone in the room heard it go.
The students now knew their score was made up. What did they think of their own ability?
There is a quieter version, and it is far more common. You do name a condition. Then the condition arrives. And you find a reason it does not really count. The sample was small. The timing was odd. This one is different. Sometimes those reasons are good ones. That is what makes this hard.
So how do you tell the two apart? Not by whether you kept the belief.
You said ten timed mornings would settle it. The mornings come out against you. Which responses are honest?
Tap an item, then a bucket
Researchers have watched people slide out of reach in a lab. They showed people facts that went against their own side. Afterwards they leaned towards calling it a matter of right and wrong. They leaned away from calling it a matter of facts. The belief did not move. It changed address.
Moving a belief out of reach of facts is comfortable. It works. The price is quiet. You keep the belief and you lose the sorting. Your good beliefs and your bad ones now feel the same. Nothing can tell them apart any more, including you.
This habit has an edge, and the edge matters. “The new route makes me late” is a claim about the world. “I want to live near my sister” is not. No timetable could ever prove it wrong, and nothing has gone missing. It was never trying to describe how things are.
The habit only works on the first kind of sentence. Asking what would disprove a wish is asking the wrong question.
Popper wrote about this himself, right after stating the rule. He had just said untestable theories are not science. Then he was careful about what that did not mean.
“I thus felt that if a theory is found to be non-scientific, or ‘metaphysical’ (as we might say), it is not thereby found to be unimportant, or insignificant, or ‘meaningless’, or ‘nonsensical’.”— Karl Popper, Conjectures and Refutations (1963)
Sorting a sentence into the right pile is half the work.
Which of these are claims about how the world is?
Tap an item, then a bucket
None of this asks you to hold nothing. Doubting everything is as cheap as believing everything. Both take no work and settle no questions. The habit here is smaller and harder. Say what would change your mind, then go and look.
You will still be sure of plenty of things. The difference is that you could say what would move you.
Plenty of people say their joints ache before the rain comes. People have said it for a very long time. It is also usually held with no condition at all. So try the question on it. What would show this is wrong?
It is a claim about the world, so an answer exists. Somebody only had to go and get it.
In 1996, two researchers published an answer. They had followed 18 arthritis patients for more than a year. Each patient named the weather they thought mattered for them. Pain got recorded, and so did the weather. They found no link they could detect.
Eighteen people is a small group. The story does not end there. In 2016, a British team asked again, using a phone app. More than 13,000 people signed up. About 2,600 sent in enough days. This time there was a link. Damp windy days with low pressure. The researchers called it real, and called it modest.
So what should a careful person believe about knees and rain? Not “I knew it all along.” Not “it was nonsense.” Something smaller than the belief people usually hold, and still open. That answer only exists because somebody named a condition and looked.
Pick one belief you hold about how something works. Write the belief. Write how sure you are out of ten. Write what would change it, and put today's date underneath. Then set a day to look. You will find out something either way. That is what the line is for.
Writing a number down will feel strange the first time. It gets easier once one date has come and gone.