Wk3 - Cautionary Tales and Dead Fish Flashcards

1
Q

What is a problem with many fMRI experiments showing correlations between brain activation and personality tests?

A

Researchers perform non-independent correlation analyses

It is not always possible to see the research methods and data analysis methods that studies used

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What in the results section might indicate that non-independent analysis has been used?

A

Very high correlations between brain activation during a task and scores on personality tests

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What in the method section might indicate that non-independent analysis has been used?

A

Brief method section

Doesn’t explain how the analysis was run

Doesn’t explain how the researcher’s chose which parts of the brain to look at

Doesn’t explain how the researcher’s corrected for multiple comparisons etc.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Who noticed puzzlingly high correlations in many papers?

A

Vul et al. (2009)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What 2 things do correlations measure?

A

Relationship between the variables

Reliability of the two measures used

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What is considered a high correlation?

A

A correlation over .8 or .9

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What does a correlation of .9 mean?

A

90% of the variance in a subject’s scores on a questionnaire can be explained by the amount of brain activity in a region. This is extremely high and rare.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

In terms of measure reliability, in what circumstances might we find a high correlation?

A

If both measures had PERFECT reliability. Rare.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Can a correlation be higher than the reliability of the scales used?

A

No

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

What is typically the highest reliability most personality scales have?

A

.8

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

What is typically the highest reliability most BOLD studies have found?

A

.7

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

If the BOLD measurement reliability is .7 and the personality measurement reliability is .8, then what is the maximum correlation a study should find?

A

r = .74

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

How is BOLD reliability calculated?

A

Test-retest of a simple task and seeing whether the same activations occur.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What makes up an fMRI image?

A

Lots of measurements of the BOLD signal in regions called voxels

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What is the range of number of voxels in ONE fMRI image?

A

40,000 - 500,000 voxels

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

How many mm of brain tissue does each voxel cover?

A

1mm3 - 125mm3

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

How often are fMRI images taken?

A

Every 2 to 3 seconds during the scan to give a time series of these images (multiple snapshots)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

How would researchers get a contrast between task conditions?

A

They would take away the activation in one condition from the activation in another condition

Compare the difference in activation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

What does a contrast between tasks yield?

A

Matrices for hundreds of thousands of numbers indicating activation levels in different voxels

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What can researchers base their choice of selecting a subset of voxels on?

A

Can select a subset of voxels based on anatomical constraints, functional constraints, or a mixture of both anatomical and functional constaints

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Give an example of selecting voxels based on anatomical constraints

A

Selecting a subset of all voxels in the amygdala

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

Give an example of selecting voxels based on functional constraints

A

Selecting a subset of all voxels that are more active to one condition than another

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

Give an example of selecting voxels based on a mixture of anatomical and functional constraints

A

Selecting a subset of all voxels that are more active to one condition in the amygdala

24
Q

Why do researcher’s have to select a subset of voxels to run further statistical analyses on?

A

To reduce the likelihood of finding significant results by chance.

Because if they run multiple tests, you can increase the likelihood on findings purely based on chance (meaningless)

If the researcher’s run a correlation analysis on hundreds of thousands of numbers, by chance they are bound to find a highly significant correlation

The more tests you run on a sample, the more likely you are to find false positives / high correlations by chance

25
Q

What is a type 1 error?

A

Finding a false positive

Finding a significant result just by chance

26
Q

How can type 1 error be reduced?

A

Choosing a subset of voxels to run the analysis on

Run less multiple tests

27
Q

Outline what Vul et al. (2009) did

A

Conducted a literature review of papers reporting correlations between BOLD signal and trait measure

Found out the exact methodology and statistical analyses the authors used (non-independent analyses)

28
Q

What did Vul et al. (2009) find?

A

High correlations in one paper (Sander et al., 2005) were the result of voxels selected on the basis of regression across subjects

29
Q

What is the process of selecting voxels based on regression across subjects?

A

Separate correlations are run between each voxel activity and the trait measure for the individual

Select the voxels which have statistically significant correlations

Report the voxel with the highest correlation or report the average correlation across the subset of voxels

30
Q

What is the name for the voxel which has the highest correlation?

A

Peak voxel

31
Q

What is cluster analysis?

A

The average correlation across the subset of voxels is reported

32
Q

What is the problem with selecting voxels based on regression across subjects?

A

The researchers are choosing the voxels on the basis of their correlation and then they are just restating this correlation

The researchers are reporting a correlation that they found across up to 500,000 separate analyses

33
Q

What is the non-independence error?

A

The non-independence error refers to the fact that the analysis that is used to select the voxels is not independent from the analysis that is then presented as the result

34
Q

How did Vul et al. (2009) illustrate the non-independence error?

A

Ran a simulation with 10 subjects, 10,000 voxels, and 1 individual difference measure

Made-up data / random numbers

Ran correlations between the made-up data for each of the voxels and the individual measure

Found high correlations in pure noise data

35
Q

What does non-independent analysis mean?

A

Voxels are selected based on a functional analysis

Researchers then report the results of that same analysis and functional data from just the selected voxels

The analysis used to select the voxels is not independent from the final analysis that is reported as the result

36
Q

Why are high correlations naturally found by chance in non-independent analyses?

A

Analysis uses the same data they used to pick the voxels in the first place

Increases the chance of high correlations (?)

37
Q

Why is there a high probability of finding a false positive correlation in fMRI experiments?

A

A large number of voxel activations are yielded by fMRI experiments

38
Q

How might the correlations from a non-independent analysis differ from an independent analysis?

A

Correlations will be stronger in a non-independent analysis

Correlations will be weaker or might not exist at all in independent analysis

39
Q

Are results from non-independent analyses meaningful?

A

No, they are just found from chance. There is just as much chance that there isn’t a real effect going on at all. The real correlation might not actually exist if independent analysis was used.

40
Q

What did Vul et al. (2009) conclude about emotion, personality, and social cognition fMRI research?

A

fMRI research uses defective research methods (53% from their literature review used non-independent analyses)

Produces numbers/results that should not be believed - personality research wants to publish exciting findings, even if they’re not true

41
Q

Why is non-independent analyses likely to be worse in fMRI studies?

A

fMRI studies contain the most numbers to look at.

There are enormous matrices of thousands and thousands of numbers corresponding to activations in different voxels.

42
Q

What might suggest use of non-independent analyses?

A

High/strong correlation - can say that the correlation might be spurious

43
Q

What can you say about strong correlations if the details of choosing voxel subset are not given in the paper’s method section?

A

Might be spurious

Worried about the strength of the correlation between a personality measure and activation because it is possible that they came by it through non-independent means

44
Q

Briefly outline the dead fish experiment

A

1 dead Salmon placed in an fMRI scanner

Shown a series of photographs of people and asked to determine what emotion the individual in the photo must have been experiencing

45
Q

What was found from the dead salmon experiment regarding the BOLD signal?

A

There was a significant increase in the BOLD signal during the photo task compared to resting state

46
Q

What was found from the dead salmon experiment regarding voxel activity?

A

Several active voxels were discovered in a cluster located within the salmon’s brain cavity

47
Q

How many voxels out of 8064 were significantly active?

A

16

48
Q

Was the dead salmon engaging in the perspective-taking task?

A

No

49
Q

What can we conclude from the dead salmon experiment about multiple comparisons and chance findings?

A

Random noise in the EPI timeseries may yield spurious results if multiple comparisons are not controlled for

Absolutely by chance, we have found activation in the dead salmon’s brain while taking part in the task compared to not taking part in the task

50
Q

What can we conclude from the dead salmon experiment regarding standard statistical thresholds and minimum cluster sizes?

A

Relying on statistical thresholds (e.g., p < .001) and low minimum cluster sizes (k > 8) is an ineffective control for multiple comparisons.

51
Q

What should fMRI studies utilise as standard practice in the computation of their statistics?

A

Should utilise multiple comparisons correction

52
Q

Why can’t we rely on less than 8 voxels in a cluster?

A

It is dangerous to call a region of activation a cluster if you have less than 8 voxels in the cluster

POINT THIS OUT IF fMRI papers report clusters where there are fewer than 8 voxels.

53
Q

What is the problem with statistics from neuroimaging studies such as fMRI studies?

A

Neuroimaging studies sometimes use dodgy/dubious statistical procedures

Brief method sections which do not describe the methods or analysis used

54
Q

What should we look at in fMRI studies?

A

Methodology

Number of voxels in a cluster

How did they choose their subset of voxels to look at?

Analysis

High correlations

55
Q

What is the amygdala important for?

A

Fear recognition

Fear conditioning

56
Q

What does the reward circuitry include?

A

NAcc

Ventral striatum

Insula

Prefrontal cortex

57
Q

Are there different circuits for positive and negative affect?

A

No, there is overlap between circuits underlying negative and positive affect