Above is a link Ignite talk about probability and Bayes’ Rule.
I gave the talk in December 2011 at Ignite Dublin at the Science Gallery, it was the low-point of a brilliant line-up organised by Conor Houghton (his last time organising Ignite Dublin: he will be greatly missed there).
The full notes are below.
This is a sequel to an Ignite talk I have previously, entitled “Probably Approximately Confused”. Very helpfully, I gave the two talks in different cities; of the couple of hundred people who saw one or the other, only two people saw both live.
The Ignite format is that talks last 5 minutes. You have 20 slides, which auto-advance every 15 seconds. The notes below are more or less exactly what I said during the talk. I have also added a couple of illustrations from my slides.
- I’m going to talk for the next few minutes about dangerous animals such as sharks
- And even more dangerous animals such as cows
- And Rev Thomas Bayes, who formulated Bayes Theorem in the 1700s
- I do research in artificial intelligence & machine learning
- We use Bayes Rule in understanding medical data
- Others use it for robot control
- Almost all spam detection software uses it too
- Everyone who works in science, engineering, finance or forecasting needs probability and Bayes Rule if they have data with uncertainty
- We use Probability to quantify the uncertainty that allows us to make rational decisions
- Defining probabilities seems simple
- You can express how likely it is for something to happen with a number in between 0-1
- But mistakes are made so commonly by professionals that they end up getting names such as the Prosecutor’s Fallacy
- I’d like to make to explain to you how to avoid making mistakes like those
- One problem we have with probabilities is mis-assigning risk:
- We are more scared of sharks than cows
- But about 1 person per year in USA killed by sharks
- While cattle kill about 20 people/yr in USA [source]
- In general, we assign high probability to high-visibility threats but unlikely threats
- This woman is concerned about jackhammer noise harming her unborn baby [source]
- But apparently not so concerned about smoking
- Of course, she’s not the only parent who makes bad estimates about relative risks
- What about parents who won’t let kids run in case they fall, walk to school because of near-zero risk of kidnapping or bogeymen, or get MMR vaccines?
- So people are naïve about probability. What should we do? Take advantage of them!
- Here’s my plan: rent out Croke Park football pitch
- Cover it with a layer of 100 million M&Ms
- Put a secret mark on the bottom of one
- People pay €2 to go in blindfolded with a tweezers
- If you pick the right one, I’ll give you half the profits
- Any takers? No? It doesn’t seem like a good deal
- And yet millions of people every week buy EuroMillions lottery tickets with worse odds
- 1 in 116,531,800
- This example shows that we are better able to appreciate probabilities when presented with hard numbers
- Let’s return to the cow and the shark
- I’m fairly rational: why is it I don’t mind letting my kids into a cow-infested grass, but I wouldn’t let them near shark-infested water?
- We can decompose the probabilities:
- P(death caused by cow) = P(cow causing death) P(meeting cow) / P(death)
- This is Bayes’ Rule!
- Likewise:
- P(death caused by shark) = P(shark causing death) P(meeting shark) / P(death)
- This captures our intuition well:
- Individual sharks are more dangerous than individual cows, but much more rarely encountered by humans
- So on aggregate cows cause more deaths
- Let’s consider the kind of problem that often appears in studies of probability:
- A randomly chosen person is tested for a disease that affects 1 in 1,000
- Get positive result from a test
- The test has 99% accuracy
- What do you think?
- Hands up if you think they probably DON’T have it
- If you put up your hand, you are right: let’s figure it out
- We saw from the M&Ms example that it’s easier if we use hard numbers: “natural frequencies”:
- So say we have 100,000 people:
- 100 will have disease, other 99,900 won’t
- The test will give a correct positive result for 99 of the 100 with disease
- Importantly, the test will give a false positive for 1% of people: 999
- It will have a correct negative result for 99%
- So we can combine the numbers
- Of all the people who get a positive result (1098), only 99 actually have cancer: 9%
- There have been a lot of research using tests like this with various professionals
- They conclude that doctors, lawyers and other professionals get these things wrong somewhere between 50% and 90% of the time
- So if you were following this,
- And if you now know to reason in terms of hard numbers when you want to work with probabilities
- You will be at the upper end of the Bell curve!
Leave a comment