Why don't we understand statistics? Fixed mindsets may be to blame

We are faced with probabilities and statistics on a daily basis. These are most commonly presented as percentages (i.e. 10% of the population), but a more intuitive way of understanding this information -- called natural frequencies -- is to present it as two whole numbers (i.e. 1 in 10 people).

Does this remind you of math problems you had to try solving in school? You're not alone.

"Even though natural frequencies are much easier to understand, people are more familiar with probabilities represented by percentages because of their education," says Patrick Weber of the University of Regensburg, Germany, who led the study with colleagues Karin Binder and Stefan Krauss.

However, although people are more familiar with probabilities, it does not mean they are any better at understanding them.

"A recent meta-analysis showed the vast majority of people have difficulties solving a task presented in probability format," says Weber. "This can result in severe misjudgments when applied in professional settings."

Weber refers to a famous example of the misuse of statistics in court when the prosecution relied heavily on flawed statistical evidence presented by a medical professional. An insufficient understanding of statistical probability led to Sally Clark being wrongly convicted of the murder of her two sons, based on the misjudgment of the probability that they could have died from natural causes.

The researchers believe that people are 'blind' to probabilities -- yet have a fear of changing them into simpler natural frequencies which would make them easier to understand.

"The same meta-analysis showed that when the task was presented in natural frequency format instead of probabilities, performance rates increased from 4% to 24%," says Weber. (See below for an example task.)

But while the success rate was much higher when the data was presented as two whole numbers rather than a percentage, around three-quarters of participants still could not solve the task at all. Weber and his colleagues were keen to find out why.

They gave groups of university students different reasoning tasks, one presented in probability format and the other in natural frequency. Participants were asked to show their working so the researchers could understand their cognitive processes behind answering the questions.

They found that, when the questions were presented in natural frequencies, half the participants did not use natural frequencies to solve the problems, but instead 'translated' them into the more difficult probability format.

Weber and his team believe that a fixed mindset -- known as the Einstellung effect -- may explain participants' preference to change the data.

"Students are a lot more familiar with probabilities than with natural frequencies due to their education. In high school and university contexts, natural frequencies are not considered as equally mathematically valid as probabilities," says Weber.

"This means that working with probabilities is a well-established strategy when it comes to solving statistical problems," Weber continues. "While in many situations students profit from such an established strategy, the mental sets developed over a long period of time during school and university can make them 'blind' to simpler solutions -- or unable to find a solution at all."

Weber and his team believe this is a widespread problem deeply rooted in school and university curricula all over the world. They do, however, recognize their study only consisted of university students which may produce different results from the general population.

"We assume that while overall solution rates might vary, the tendency to avoid using natural frequencies is widespread across the whole population," says Weber.

The researchers hope their new insights -- published in a research collection on judgment and decision making under uncertainty -- will encourage global change to statistical teaching strategies in schools and universities.

"We want our findings to encourage curriculum designers to incorporate natural frequencies systematically into school mathematics and statistics. This would give students a helpful tool to understand the concept of uncertainty -- in addition to the 'standard' probabilities."

Example of a problem posed in probability and natural frequency format

Probability format: The probability of being addicted to heroin is 0.01% for a person randomly picked from a population (base rate). If a randomly picked person from this population is addicted to heroin, the probability is 100% that he or she will have fresh needle pricks (sensitivity). If a randomly picked person from this population is not addicted to heroin, the probability is 0.19% that he or she will still have fresh needle pricks (false alarm rate). What is the probability that a randomly picked person with fresh needle pricks is addicted to heroin (posterior probability)?

Solution: With the help of Bayes' theorem, the corresponding posterior probability P(H|N), with H denoting "person is addicted to heroin" and N denoting "person has fresh needle pricks," can be calculated:

P(H|N) = (P(N|H) x P(H)) / (P(N|H) x P(H) + P(N|¬H) x P(¬H)) = (100% x 0.01%) / (100% x 0.01% + 0.19% x 99.99%) = 5%

Natural frequencies format: 10 out of 100,000 people from a given population are addicted to heroin. 10 out of 10 people who are addicted to heroin will have fresh needle pricks. 190 out of 99,990 people who are not addicted to heroin will nevertheless have fresh needle pricks. What percentage of the people with fresh needle pricks is addicted to heroin?

Solution:

Number of heroin addicts: 10

Number of people with needle pricks: All the heroin addicts + 190 non-addicts = 200

Percentage of people with needle pricks who are addicts = 10/200 = 5%