Non-fic: Black Box Thinking: Matthew Syed Flashcards
‘Part 1: The Logic of Failure’
What is the accident ratio for flights on western-built jets?
How many deaths per year roughly?
Chapter 1: A Routine Operation
One fatal accident per 2 - 4 million flights depending on the year.
210 deaths in 2013 (Year divided by ten roughly)
‘Part 1: The Logic of Failure’
What is the third biggest killer in the United States?
Chapter 1: A Routine Operation
Preventable medical errors
‘Part 1: The Logic of Failure’
Explain the concepts closed loop/open loop
Chapter 1: A Routine Operation
A closed loop as where failure doesn’t lead to progress because information on errors and weaknesses is misinterpreted or ignored; an open loop does lead to progress because the feedback is rationally acted upon
‘Part 1: The Logic of Failure’
What key pressure prevented staff on United Airlines 173 and those treating Elaine Bromily from taking a further stand which would have resulted in catastrophe being averted?
Chapter 2: United Airlines 173
… both the engineer and the nurse were intimidated by the sense of hierarchy. Social pressure, and the inhibiting effect of authority, had destroyed effective teamwork
**‘Part 1: The Logic of Failure’
**
What is the key difference between the way that the medical profession and the aviation industry respond to errors?
Chapter 2: United Airlines 173
Aviation is data rich and transparent. Challenge is welcomed. The medical profession, in comparison, is data poor, and criticism is seen as personal.
**‘Part 1: The Logic of Failure’
**
How is a plane journey like a hypothesis?
Chapter 2: United Airlines 173
… Because we postulate that in this condition, with these pilots, and these specific circumstances all will run safely.
If it fails to do so, the hypothesis has been falsified. This is an opportunity to learn from failure… Just as the scientific method improves in general terms through falsification
**‘Part 1: The Logic of Failure’
**
Why are psychotherapists like golfers playing at night?
Chapter 3: The Paradox of Success
Because they are playing ‘in the dark’.
They rely on subjective data which is highly unreliable to ascertain whether or not they have been successful.
Like playing golf at night… feedback is crucial for improvement.
And data is often ‘time lagged’ (unlike practising a sport)
‘Part 1: The Logic of Failure’
What are the (three) essential elements for using feedback to generate ‘theories of improvement’?
Chapter 3: The Paradox of Success
One: that they are falsifiable
Two: that feedback is quickly accessible
Three: that the prevailing culture values and accepts feedback
‘Part 1: The Logic of Failure’
What is the dangerous attitude to error that we should seek to avoid?
Chapter 3: The Paradox of Success
That error arises from a lack of compassion / talent / intention to do good
The biggest danger is the stigmatisation of error
‘Part 2: Cognitive Dissonance’
How do prosecutors often react to DNA evidence which proves a conviction was wrongly given?
Chapter 4: Wrongful Convictions
By inventing spurious reasons why the evidence in fact fits, such as pretending that an entirely new man was on the scene and then disappeared (cases of sexual violence often postulate an ‘unindited co-ejaculator’). This often violates the original premise of the case.
Chapter 4: Wrongful Convictions
‘Part 2: Cognitive Dissonance’
What emotional response is often employed when we discover we are very wrong? (In relation to evidence)
Chapter 4: Wrongful Convictions
We tend to reframe the evidence.
When confronted with evidence that challenges our deeply held beliefs we are more likely to reframe the evidence than we are to alter our beliefs.
Denial
‘Part 2: Cognitive Dissonance’
What was the outcome when Charles Lord (a psychologist) gave two groups of diametrically opposed people well researched reports on capital punishment arguing both sides?
Chapter 4; Wrongful Convictions
Rather than seeing merit in both sides, the groups became even further apart and saw research that supported their ideas positively whereas the research which challenged their ideas they regarded as fundamentally flawed (even though those reports were well researched and high quality)
‘Part 2: Cognitive Dissonance’
What was your take away from the section of Black Box thinking called ‘Cognitive Dissonance’?
That intense emotional involvement with an area where they have close personal association is likely to lead to cognitive dissonance when failure occurs… We need to depersonalise in order to have a chance of people coming to clear judgements and learning from errors
‘Part 2: Cognitive Dissonance’
Why do good people, like doctors, engage in systematic cognitive dissonance?
Chapter 5: Intellectual Contortions
Because they are good people, and the challenge to self-identity would otherwise be unbearable.
‘Part 2: Cognitive Dissonance’
How did Blair’s statement around Iraq war show CD?
Chapter 5: Intellectual Contortions
There are WMD. We will find WMD. WMD are hidden. WMD have been destroyed or hidden. Things are bad in Syria (where we didn’t intervene). I cannot apologise for removing Saddam.
At each point, conviction growing stronger not weaker (a feature of CD).
‘Part 2: Cognitive Dissonance’
Ironically, the more famous the expert, the less accurate his or her predictions tended to be.
Why is this?
Chapter 5: Intellectual Contortions
Cognitive dissonance. Those most publicly associated with their predictions are most likely NOT to change their minds (or learn from their mistakes). They have the most to lose when things go wrong, but don’t learn lessons.
‘Part 2: Cognitive Dissonance’
Intelligence and seniority, when linked to CD becomes..?
Chapter 5: Intellectual Contortions
… one of the most formidable barriers to progress in the world today.
Ego is dangerous.
How did Trofim Lysenko introduce dangerous thinking in Soviet Russia?
Upon which 19th century scientist did he base his work?
Chapter 6: Reforming Criminal Justice
1934 - he pushed non-scientific ideas; traits acquired during one’s lifetime could be passed on. He applied this to crop yields (the notion that through close planting, crops from the same species would not compete with each other). Climate of fear meant that these disastrous ideas were non-falsifiable.
(Lamarckism)
What did the paper ‘extraneous factors in judicial decisions’ from the Israeli National Academy of science reveal?
Chapter 6; Reforming Criminal Justice
Judges are much less likely to grant parole when they are hungry.
From a 65% chance of parole to a 0% chance.
In scientific change - like The Industrial Revolution- we sometimes have progress backwards.
How does the development of the steam engine pump demonstrate this?
(Clue: new ways captivate)
Chapter 7: Confronting Complexity
The first steam engine pumping water was built by Thomas Newcomen who was barely literate. It was developed further by James Watt. The success of the engine raised a deep question: why does this incredible device actually work (it broke to the laws of physics)?
This question inspired Nicholas Leonard Sadi Carnot - the French physicist who developed the laws of thermodynamics.
Explain the narrative fallacy
Chapter 7: Confronting Complexity
The tendency to narrate the reasons for things after the event so that they are neat and explicable / seemingly predictable (in hindsight)
What does the narrative fallacy push us away from?
Chapter 7: Confronting Complexity
Underlying complexity / hard to see variables / unforeseeable interactions
Why don’t the Unilever biologists mind failure?
Chapter 7: Confronting Complexity
Because it is built into their planning – it is intentional (rather than a personal failing)
Explain the MVP prototype model
Chapter 7: Confronting Complexity
Minimum Viable Product- Sufficient similarity to final intended product that it can be tested on early adopters and refined at low expense. This avoids the perfectionism and resource intensive ‘top down’ model (planning a perfect product using just your own knowledge / expertise).
What is the key insight from the Nozzle chapter about narrative vs testing?
Chapter 7: Confronting Complexity
Clinging to narratives rather than testing and adapting prevents growth / improvement. Bottom up beats top down.
What is ‘counter-factual’ thinking?
Chapter 8: Scared Straight
Thinking about what did not happen (and so is ‘counter’ to the facts).
What is the key difficulty with the ‘counter-factual’?
Consider the example of OSL.
Chapter 8: Scared Straight?
You can’t observe it. You don’t know what would have happened if you didn’t do something.
So, if you run OSL and results improve … you still don’t know what would have happened without running it. You don’t know what potentially hidden variables caused the improvement or whether you would have still improved (or improved further) without it.
What do RCTs do? For OSL..?
Chapter 8: Scared Straight?
Randomised Control Tests filter out the variable you are measuring to see what happens without it.
You might look at the improvement of pupils who DID NOT attend OSL to measure their improvement.
What is the story of the ‘Scared Straight’ Programme?
Chapter 8: Scared Straight?
It’s a USA programme that suggested - appealingly - that a visit to a harsh prison would deter future crime. Reports on its effectiveness suggested it worked and it carried on for many years. However, randomised control tests revealed it actually made criminal behaviour in future MORE likely. Despite this, its appeal and cognitive dissonance allowed the project to continue for decades in several countries until it was finally closed after a huge meta analysis.
Just asking ‘did this work?’ is not proper data gathering.
“ Ultimately the approach emerges from a basic property of empirical evidence: to find out if something is working, you must -
i…… its e….. .”
Chapter 8: Scared Straight?
- isolate its effect. “
How does sports use each failure as an opportunity for marginal gains?
Chapter 9: Marginal Gains
By gathering a huge amount of data on very small elements of the process, analysing that data carefully and making small tweaks to gradually improve.
Charles Nemeth of UoC found that when trying to solve problems and stimulate creative ideas…?
(Link to ‘debate culture’)
… the encouragement of debate – and even criticism if warranted – appears to stimulate more creative ideas. Cultures that permit and even encourage such expression of differing viewpoints may stimulate the most innovation.
