Expert judgment

Question 1

What are 4 different ways of identifying an expert?

Answer 1

1. Let journalists and media identify the experts
2. The person with the most experience is the expert
3. Let the expert identify themselves
4. Look for a certain psychological profile

Question 2

What is the most important cost estimation technique?

Answer 2

Expert judgment, since a lot of the other cost estimation methods use expert judgment. Bottom up, Analogous, three-point, reserve, group decision making technique

Question 3

What is an expert?

Answer 3

Someone who has a high knowledge or skill in a subject or activity.

Question 4

What is an epistemic expert?

Answer 4

An expert who their expertise is a function of what they know. Someone who can bring strong justifications for a range of propositions in a domain. Can teach the subject

Question 5

What is a performative expertise?

Answer 5

Those who their expertise is a function of what they do. Can do the project. PPerformative expertise is the capacity to perform a skill well according to the rules and virtues of a practice.

Question 6

What is an expert opinion?

Answer 6

A claim made by an expert, the expert provides a strong justification for it, the claim is in the domain of the experts expertise

Question 7

What are the two definitions of expert judgment?

Answer 7

PMBOK: The gut feeling offered by the person with the most experience in the room

Weinstein: Expert opinon is an opinion with strong justifications, within the knowledge domain of an expert

Question 8

What are the issues with the 10 000 hour rule?

Answer 8

Gladwell based his saying about 10k hour rule on a study done by Ericsson on violinists that said that the average student had studied 10k hour rule by the time they were 20. This study does not show anything about how many hours they woud need in total to become professionals in their domain, neither does it show how many hours you would need for other domains.

Also the practice cannot be naive, it needs to be deliberate practice.

It also implies that anyone can become an expert if they put in the 10k hour, this isn’t neccessary true, since it also can depend on their personal abilities. The study was done on violin students who as already good enough to go to music academy. So we cannot say that deliberate practice will be enough.

The main point of 10k hour rule is to become accomplished in any field you would need to put in the hours and work for it, and you will need to do it with great effort. This is more important when working in a field where there is a great history of people working to become experts.

Question 9

What is meant by deliberate practice?

Answer 9

1. Good teacher: Is about getting the skills that other people already have figured out.
2. Pushing yourself: Need to try things that are currently beyond their abilities
3. Work on specific goal: Can use the SMART framework
4. Needs full attention and concentration: not enough to just follow the teachers words
5. Requires feedback: Doctors who did not get any feedback, got worse with more experience
6. Developing mental representations: Which helps in monitoring your progress and notice when something is being done wrong.
7. Continiously keep revise and improve what you have learned.

Question 10

Does IQ matter?

Answer 10

According to Tetlock, IQ did not matter when it came to the accuracy of predictions/forecasting done by experts. We’ve also seen that cognitive diversity is more important than IQ.

Question 11

Why is it important to have grit as an expert?

Answer 11

To be able to spend many deliberate practice hours on a field of domain, you need grit. Grit is the ability to work hard and long towards a goal.
It can be divided into passion and preservation

Question 12

What is meant by growth mindset?

Answer 12

The opposite of fixed mindset. It means that you think you have the capability to change, no matter what your starting point is. It is important in order to have grit.

Question 13

What is an expert?

Answer 13

You need to do a lot of deliberate practice over a long period of time, to do so you need grit. To have grit you often need growth mindset, interest and purpose.

Question 14

Why does weinstein think that experts are more likely to give a true opinion?

Answer 14

Experts can give a strong justification over a range of claims in a domain, and they usually have better access to justificatory information. He thinks that the strength of justification is linked to the likelihood of it being true.

This is not backed up by any research.

Question 14

Why is it difficult to determine if experts are good at forecasting?

Answer 14

They usually forecast to vague for us to check the accuracy of it afterwards.

Question 15

What does Tetlock say about experts abilities to forecast accurate?

Answer 15

His research shows that the average expert got a brier-score of 0.5, that is as accurate as a dart throwing chimpanzee.

He also saw that the more famous an expert was, the higher was his brier score, less accurate was his predictions.

Question 16

What is best of clinical predictions or statistical predictions?

Answer 16

It has been shown through many rereashes that statistical predictions are in average better than clinical predictions. This has both Meehl and Kahneman shown.

Question 17

What are four ways to identify an expert?

Answer 17

• Let the media identify them
• Self identification
  -The one with the most experience
• Find the one with correct pshychological profile

Question 18

Is it a good idea to let the media identify them?

Answer 18

No, because we have seen that when in need for an expert, the media will assign one, even though they may not have any expertise in the relevant domain.

We’ve also seen from the research of Tetlock that the more famous the expert was, the less accurate was his predictions. Which means that the experts the media uses frequently, had a higher brier score.

Tetlock explains this with journalists being more concerned with telling a good story rather than being truthful and accurate.

Question 19

Is it good to let the amount of experience choose the expert?

Answer 19

Quite often we ee that experience doesn’t improve performance. According to Ericsson the more experience doctors got, the worse they got.

But according to Kahneman they need to be in a stable environment with feedback for their experience to improve performance.

For experience to matter, we need to be able to control for quality.

Question 20

Why isn’t self identification the way of identifying experts?

Answer 20

• Dunning-Kruger
• Overconfidence

Question 21

Can we identify experts based on their psychological profile?

Answer 21

Foxes rather than hedgehogs. Those who were below the average on the forecasting accuracy, while foxes were above the average.

Question 22

What is the difference between foxes and hedgehogs?

Answer 22

The hedgehogs were quite deterministic, and did not change their mind even when their predictions failed. They were unusually confident.

Foxes gathered a lot of information from different sources, and talked about probabilities rather than certainties. They were more likely to admit that they were wrong if the prediction failed.