Wk3 - Cautionary Tales and Dead Fish Flashcards

1
Q

What is a problem with many fMRI experiments showing correlations between brain activation and personality tests?

A

Researchers perform non-independent correlation analyses

It is not always possible to see the research methods and data analysis methods that studies used

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What in the results section might indicate that non-independent analysis has been used?

A

Very high correlations between brain activation during a task and scores on personality tests

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What in the method section might indicate that non-independent analysis has been used?

A

Brief method section

Doesn’t explain how the analysis was run

Doesn’t explain how the researcher’s chose which parts of the brain to look at

Doesn’t explain how the researcher’s corrected for multiple comparisons etc.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Who noticed puzzlingly high correlations in many papers?

A

Vul et al. (2009)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What 2 things do correlations measure?

A

Relationship between the variables

Reliability of the two measures used

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What is considered a high correlation?

A

A correlation over .8 or .9

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What does a correlation of .9 mean?

A

90% of the variance in a subject’s scores on a questionnaire can be explained by the amount of brain activity in a region. This is extremely high and rare.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

In terms of measure reliability, in what circumstances might we find a high correlation?

A

If both measures had PERFECT reliability. Rare.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Can a correlation be higher than the reliability of the scales used?

A

No

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

What is typically the highest reliability most personality scales have?

A

.8

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

What is typically the highest reliability most BOLD studies have found?

A

.7

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

If the BOLD measurement reliability is .7 and the personality measurement reliability is .8, then what is the maximum correlation a study should find?

A

r = .74

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

How is BOLD reliability calculated?

A

Test-retest of a simple task and seeing whether the same activations occur.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What makes up an fMRI image?

A

Lots of measurements of the BOLD signal in regions called voxels

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What is the range of number of voxels in ONE fMRI image?

A

40,000 - 500,000 voxels

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

How many mm of brain tissue does each voxel cover?

A

1mm3 - 125mm3

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

How often are fMRI images taken?

A

Every 2 to 3 seconds during the scan to give a time series of these images (multiple snapshots)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

How would researchers get a contrast between task conditions?

A

They would take away the activation in one condition from the activation in another condition

Compare the difference in activation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

What does a contrast between tasks yield?

A

Matrices for hundreds of thousands of numbers indicating activation levels in different voxels

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What can researchers base their choice of selecting a subset of voxels on?

A

Can select a subset of voxels based on anatomical constraints, functional constraints, or a mixture of both anatomical and functional constaints

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Give an example of selecting voxels based on anatomical constraints

A

Selecting a subset of all voxels in the amygdala

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

Give an example of selecting voxels based on functional constraints

A

Selecting a subset of all voxels that are more active to one condition than another

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

Give an example of selecting voxels based on a mixture of anatomical and functional constraints

A

Selecting a subset of all voxels that are more active to one condition in the amygdala

24
Q

Why do researcher’s have to select a subset of voxels to run further statistical analyses on?

A

To reduce the likelihood of finding significant results by chance.

Because if they run multiple tests, you can increase the likelihood on findings purely based on chance (meaningless)

If the researcher’s run a correlation analysis on hundreds of thousands of numbers, by chance they are bound to find a highly significant correlation

The more tests you run on a sample, the more likely you are to find false positives / high correlations by chance

25
What is a type 1 error?
Finding a false positive Finding a significant result just by chance
26
How can type 1 error be reduced?
Choosing a subset of voxels to run the analysis on Run less multiple tests
27
Outline what Vul et al. (2009) did
Conducted a literature review of papers reporting correlations between BOLD signal and trait measure Found out the exact methodology and statistical analyses the authors used (non-independent analyses)
28
What did Vul et al. (2009) find?
High correlations in one paper (Sander et al., 2005) were the result of voxels selected on the basis of regression across subjects
29
What is the process of selecting voxels based on regression across subjects?
Separate correlations are run between each voxel activity and the trait measure for the individual Select the voxels which have statistically significant correlations Report the voxel with the highest correlation or report the average correlation across the subset of voxels
30
What is the name for the voxel which has the highest correlation?
Peak voxel
31
What is cluster analysis?
The average correlation across the subset of voxels is reported
32
What is the problem with selecting voxels based on regression across subjects?
The researchers are choosing the voxels on the basis of their correlation and then they are just restating this correlation The researchers are reporting a correlation that they found across up to 500,000 separate analyses
33
What is the non-independence error?
The non-independence error refers to the fact that the analysis that is used to select the voxels is not independent from the analysis that is then presented as the result
34
How did Vul et al. (2009) illustrate the non-independence error?
Ran a simulation with 10 subjects, 10,000 voxels, and 1 individual difference measure Made-up data / random numbers Ran correlations between the made-up data for each of the voxels and the individual measure Found high correlations in pure noise data
35
What does non-independent analysis mean?
Voxels are selected based on a functional analysis Researchers then report the results of that same analysis and functional data from just the selected voxels The analysis used to select the voxels is not independent from the final analysis that is reported as the result
36
Why are high correlations naturally found by chance in non-independent analyses?
Analysis uses the same data they used to pick the voxels in the first place Increases the chance of high correlations (?)
37
Why is there a high probability of finding a false positive correlation in fMRI experiments?
A large number of voxel activations are yielded by fMRI experiments
38
How might the correlations from a non-independent analysis differ from an independent analysis?
Correlations will be stronger in a non-independent analysis Correlations will be weaker or might not exist at all in independent analysis
39
Are results from non-independent analyses meaningful?
No, they are just found from chance. There is just as much chance that there isn't a real effect going on at all. The real correlation might not actually exist if independent analysis was used.
40
What did Vul et al. (2009) conclude about emotion, personality, and social cognition fMRI research?
fMRI research uses defective research methods (53% from their literature review used non-independent analyses) Produces numbers/results that should not be believed - personality research wants to publish exciting findings, even if they're not true
41
Why is non-independent analyses likely to be worse in fMRI studies?
fMRI studies contain the most numbers to look at. There are enormous matrices of thousands and thousands of numbers corresponding to activations in different voxels.
42
What might suggest use of non-independent analyses?
High/strong correlation - can say that the correlation might be spurious
43
What can you say about strong correlations if the details of choosing voxel subset are not given in the paper's method section?
Might be spurious Worried about the strength of the correlation between a personality measure and activation because it is possible that they came by it through non-independent means
44
Briefly outline the dead fish experiment
1 dead Salmon placed in an fMRI scanner Shown a series of photographs of people and asked to determine what emotion the individual in the photo must have been experiencing
45
What was found from the dead salmon experiment regarding the BOLD signal?
There was a significant increase in the BOLD signal during the photo task compared to resting state
46
What was found from the dead salmon experiment regarding voxel activity?
Several active voxels were discovered in a cluster located within the salmon's brain cavity
47
How many voxels out of 8064 were significantly active?
16
48
Was the dead salmon engaging in the perspective-taking task?
No
49
What can we conclude from the dead salmon experiment about multiple comparisons and chance findings?
Random noise in the EPI timeseries may yield spurious results if multiple comparisons are not controlled for Absolutely by chance, we have found activation in the dead salmon's brain while taking part in the task compared to not taking part in the task
50
What can we conclude from the dead salmon experiment regarding standard statistical thresholds and minimum cluster sizes?
Relying on statistical thresholds (e.g., p < .001) and low minimum cluster sizes (k > 8) is an ineffective control for multiple comparisons.
51
What should fMRI studies utilise as standard practice in the computation of their statistics?
Should utilise multiple comparisons correction
52
Why can't we rely on less than 8 voxels in a cluster?
It is dangerous to call a region of activation a cluster if you have less than 8 voxels in the cluster POINT THIS OUT IF fMRI papers report clusters where there are fewer than 8 voxels.
53
What is the problem with statistics from neuroimaging studies such as fMRI studies?
Neuroimaging studies sometimes use dodgy/dubious statistical procedures Brief method sections which do not describe the methods or analysis used
54
What should we look at in fMRI studies?
Methodology Number of voxels in a cluster How did they choose their subset of voxels to look at? Analysis High correlations
55
What is the amygdala important for?
Fear recognition Fear conditioning
56
What does the reward circuitry include?
NAcc Ventral striatum Insula Prefrontal cortex
57
Are there different circuits for positive and negative affect?
No, there is overlap between circuits underlying negative and positive affect