CHAP 12 : Strengths and Weaknesses of AI

Question 1

What are 3 weaknesses of humans, as compared to AI?

Answer 1

1. Humans are poor statisticians
2. Framing problems
3. Inconsistent

Question 2

Wha is meant by humans are poor statisticians?

Answer 2

• Humans are often unable to assess probabillities of events properly
• even with accurate probabilities provided, they make poor decisions

e.g. from lect notes. (see notes for another e.g.)
after seeing a biased coin tossed five times and the outcome is HTHHT, humans often predict randomly with 0.6 probability of head when asked to guess the next few tosses, but the prediction to maximise accuracy is guessing all H.

Question 3

What is meant by humans having framing problems?

Answer 3

• Humans often make different predictions with the same information, depending on how the question is asked

e.g from lect notes :
When physicians were presented with 2 choices to treat cancer : radiation/ surgery

• When group 1 was told “the one month survival rate [for surgery] is 90%, 84% of physicians opted to treat patients w surgery
• When group 2 was told “there is a 10% mortality in the first month”, 50% of physicians opted to treat patients with surgery

Question 4

What does humans being inconsistent in prediction mean?

Answer 4

• Humans sometimes predict outcomes inconsistently despite being given the same info

Question 5

Why do humans still do better in many scenarios as compared to AI?

Answer 5

• current AIs are mainly narrow AI and general artificial intelligence
• where current AI is designed to do a single specific task and any knowledge learnt from that task will not automatically applied to other tasks

Question 6

What are the 3 tasks that humans are still better at, compared to AI?

Answer 6

1. Learning from a small amount of data
2. Causal inferences
3. Estimating utility

Question 7

What are some examples of tasks which demonstrates human can learn from a small amount of data?

Answer 7

• Playing a new game after a small number of trials
• cooking using a new recipe from watching a small number of demonstrations

Question 8

Why are humans better at learning from a small amount of data than AI?

Answer 8

• Humans can integrate their experience throughout their life, allowing them to learn from small amounts of data while machine learning is done in isolation on task of interest
• Evolution may have pre-built knowledge in humans, allowing faster learning from a smaller amount of data

-HOWEVER, AI techniques often require large amt of training data for performance,

Question 9

Humans often have better causal knowledge of the world than machines? True or False? Give an example

Answer 9

True.

E.g. from slides
- Early chess program that was trained from grandmaster games
- The program found an association between sarificing the queen and winning and thought these events shared a causal r/s , so when it plays the game, it always sacrificed the queen
- This has resulted the chess program to lose to humans, as association does not imply causation! The cause of winning the game was due to the state of the game, and not sacrificing the queen under all contexts.

Question 10

What is Simpson’s pardox?

Answer 10

The phenomenon where association between 2 variables disappears or reverses itself when the population is divided into subpopulations

Question 11

Humans are able to use general knowledge to better infer the direction of any causal relationships and to infer the presence of any unobserved (confounding) variables. True or False?

Answer 11

True

Question 12

Why are humans better at estimating utility than AI? Give an example.

Answer 12

• Humans have the ability to incorporate subjective factors, such as personal preferences and emotional reactions, into their decision-making process, whereas AI systems are typically limited to objective data and mathematical calculations.
• e.g. consider a situation where a person is trying to decide whether to take a new job offer or stay in their current position. While an AI system could analyze objective factors like salary, benefits, and commute time, it may not be able to fully account for subjective factors like job satisfaction, career goals, and work-life balance.

Question 13

List 2 scenarios where human judgement is usually required in ML.

Answer 13

1. In developing machine learning methods.
   e.g. to specify the objective function
2. To decide between classfiers (e.g. specificity, true neg rate and sensitivity, true positive rate) to evaluate model’s performance.

• for example, if one classifier has a higher precision but lower recall than the other –> which classifier’s performance is better? Should we base it off on higher precision or recall?