Lecture 11: Group Level Analyses and Statistics

Question 1

In most studies we want to be able to say something general

Answer 1

about our whole sample of participants

Question 2

In most studies, we want to be able to say something general about our whole sample of participants

group level analyses in sensor space

to do this we can

Answer 2

average across individuals

Question 3

In most studies, we want to be able to say something general about our whole sample of participants

group level analyses in sensor space

To do this we average across individuals

This helps us to

Answer 3

reduce noise and get a clearer picture of the brain’s response, and visualise our effects

Question 4

In most studies, we want to be able to say something general about our whole sample of participants

To do this we average across individuals

This helps us to reduce noise and get a clearer picture of the brain’s response, and visualise our effects

We also conduct in group level analyses in sensor space

Answer 4

statistical tests to make comparisons, e.g., between conditions of an experiment.

We can do this with the different types of results we have, both in sensor space, e.g., below..

Question 6

Group level analyses in sensor space is easy than group level analyses in source space as

Answer 6

sensor 1 is same across participants

Question 7

We can do group level analyses in (2)

Answer 7

1. Sensor space
2. Source space

Question 8

We can do group level analyses in source space

Question 9

Diagram of group level analyses in source space across participants- (3)

Answer 9

• Activity map at a single frequency –> alpha
• Activity map at a single timepoint
• ROI time course

Question 10

We can do group analyses in source space where we can also

Answer 10

OR oull out ROI (Scout) time coruses and do statististics on these values

Question 11

Group level analyses must be transformed into

Answer 11

shared MNI space

Question 12

Trasnforming MEG source data into group soace is useful as

Answer 12

able to average source localised data across participants in a common coordinate space (e.g., MNI or a group-averaged brain)

Question 13

How does it work by transforming MEG source data into group space?

Answer 13

Works by inflating each hemisphere and then aligning them to a template (easier to align spheres than folding patterns)

Question 14

Trasnforming MEG source data into group space allows us to do

Answer 14

This allows us to do group-level visualisation and statistics
in source space

Question 15

Group level statistics our statistical tests might be - (2)

Answer 15

1. Parameteric
2. Non-parametric

Question 16

In parametric there is in terms of assumptions

Answer 16

stronger assumptions, including normally distributed data)

Question 17

Parametric tests for group level statistics is ( 2)

Answer 17

Usually t-tests

Statistical significance is then calculated based on the distribution of the test statistic

Question 18

Group leve statistics for non-parametric that have fewever assumptions (3)

Answer 18

Including non-parametric versions of standard tests e.g., Mann Whitney U-test

Safer as neuroimaging data may not be normally distributed (esp when doing many tests)

But less power

Question 19

For group level statistics instead of parametric or non-parametric we can do a resampling based approach (2)

Answer 19

Includes permutation and boot-strapping methods

Avoids assumptions about the form of the data, and is therefore non-parametric and quite robust

Question 20

In resmapling based approaches for group-level statistics - (7)

Answer 20

Say we want to compare values for conditions A and B

We calculate our test statistic as normal, e.g., a t-statistic

But then we build a null distribution using our data

On each resampling iteration, we scramble the group membership of the data, and recalculate the test

The distribution of these resampled statistics will still center on 0 (i.e., no difference) but doesn’t have to be parametric - normally distributed

We calculate a p-value by comparing our original t-statistic to the distribution of the resampled t-statistics

Increasingly popular but much slower to run

Question 21

How do we apply the group-level statistics?

Answer 21

• Test is repeated independently at each time point/sensor/location/freuency

Question 22

How to apply group-level statistics

The test is repeated independently at each time point/sensor/location/frequency

E.g., When compare the time course of activity in two conditions + a single condition - (3)

Answer 22

In each participant, subtract the time courses for each condition

Compare the difference to 0 with a t-test at each time point at the group level

For a single condition, simply compare to 0 without a subtraction

Question 23

How to apply group-level statistics

The test is repeated independently at each time point/sensor/location/frequency

E.g., for sensors - (3) same as each time point

Answer 23

Same for topographies/current density maps across the whole head

Calculate difference and compare to 0 or compare single condition to 0

Do t-tests across the group at each sensor/vertex

Question 24

How do we apply group level statistics for multivariate analyses - (2)

Answer 24

For multivariate analyses, decoding accuracy over time would be tested with a t-test vs. chance at each time point

Time-frequency plots in two conditions would be compared with a t-test at each time and frequency pair

Question 25

Problem time for neuroimagning data - (7)

Answer 25

Neuroimaging data are ‘big’ – they involve measurements at
many spatial locations and points in time

In an MEG study we might easily have something like (248 sensors, 1500 time points, Perhaps six frequency bands, or 50 or 100 different frequencies)

If we multiply these together, we are quickly making millions of comparisons!

That is 375,000 t-tests in an event-related sensor space analysis

Potentially even more in source space (e.g., 15,000 vertices * 1,500 time points * 6 frequency bands)

And that’s just for a single condition – we have three conditions and three contrasts between them

Could be 6 contrasts * 15,000 vertices * 1,500 time points * 50 frequencies…

Question 26

MEG multiple comparison problem - (7)

Answer 26

The more tests we run, the greater our chances of getting a false positive (also called a Type 1 error)

This is where we get a significant effect, even though there is no true difference

Likelihood is known as the familywise error rate, given by: FWER = 1 - (1 - ⍺)m

Where ⍺ is the threshold for significance (e.g., 0.05) and m is the number of tests

This plot is just for 100 tests and the FWER approaches 1

Clearly, we will end up with some false positives if we have 1000 or 1 million comparisons

We have no way to tell which effects are real and which are errors

Question 27

How to solve MEG multiple comparison problem - (3)

Answer 27

Bonferroni correction (Adjust the threshold for significance)

False discovery rate correction (Accept that we will make errors, but control their rate)

Cluster correction (Take into account correlations across space/time/frequency)

Question 28

In bonferroni correction - (6)

Answer 28

We adjust the threshold for significance (⍺) by dividing it by the number of tests

If we have 8 tests, ⍺ = 0.05/8 = 0.0063

Count a test as significant if its p-value is lower than this

This keeps the familywise error rate at or below ⍺ (i.e., still 5% likelihood of a false positive)

Very conservative

Dramatically reduces our ability to detect true effects (aka power) so we need a very large sample size

Question 29

In false discovery rate (FDR) - (5)

Answer 29

Less conservative – more power

The familywise error rate is the proportion of false positives expected across all tests

The false discovery rate is the proportion of false positives expected across all significant tests

We fix the FDR at a known level (e.g., 0.05), meaning we accept that there will be some number of false positives

But we don’t let them go crazy e.g., 5% of all our significant results might be false but no more

Question 30

How does FDR work? - (5)

Answer 30

We rank order the list of p-values from all tests

Set the FDR e.g., 0.05, and use to make different significance thresholds for the lowest p-value, and the second lowest, etc.

Compare each p-value to the significance threshold for that list position

our smallest p values judged against harsher threshold

Our biggest p values are judged against a lazer threshold

Question 31

In MEG cluster correction is different from FDR and bonferroni as they assume .. which is not the case.. (3)

Answer 31

our different tests are independent, and they are not (at all!)

Real effects are likely to extend over several contiguous time points, frequencies, sensors, or brain locations

Also our sampling is arbitrary – we can sample the cortex with 1500 vertices or 150,000 (same goes for time and frequency)

Question 32

In MEG cluster correction it - (4)

Answer 32

• Cluster correction identifies ‘clusters’ = contiguous samples that are all individually significant without correction
• Controls the false positive rate at this cluster (not individual test) level
• Takes into account the correlations across space, time and frequency inherent in MEG signals
• Gives better statistical power

Question 33

Steps of cluster correction - identyfing clusters - (2)

Answer 33

1. Perform a test at each sample (usually a t-test)
2. Identify clusters of adjacent significant tests

Question 34

After identifying clusters in MEG cluster correction need to

Answer 34

choose significant clusters

Question 35

Steps of choosing significant clusters - (4)

Answer 35

1. In each cluster, sum the t-statistic across all samples
2. Find the cluster with the largest summed t-statistic
3. Generate a null distribution (the blue histogram) by resampling the largest cluster’s data with random condition labels – on each resample recalculate the summed t-statistic
4. Compare each cluster to this null distribution Those with a summed t-statistic falling outside the 95% limits are retained as significant

Question 36

Summary of all steps of meg cluster correction - (6)

Answer 36

1. Perform a test at each sample (usually a t-test)
2. Identify clusters of adjacent significant tests
3. In each cluster, sum the t-statistic across all samples
4. Find the cluster with the largest summed t-statistic
5. Generate a null distribution (the blue histogram) by resampling the largest cluster’s data with random condition labels – on each resample recalculate the summed t-statistic
6. Compare each cluster to this null distribution Those with a summed t-statistic falling outside the 95% limits are retained as significant

Question 37

Pros of cluster correction - (3)

Answer 37

+ Maintains statistical power and doesn’t ‘shrink’ clusters
+ Takes correlations across time/space/frequency into account
+ Non-parametric

Question 38

Cons of cluster correction - (2)

Answer 38

• It is quite slow because of the resampling
• Can’t make strong claims about the edges of a cluster e.g., the significant start and end times in a time course (as wouldn’t necessarily be significant after correction on their own) i.e., might ‘grow’ clusters

Question 39

Summary of group level analyses - (6) and statistics

Answer 39

Get group averages to visualise MEG results

Run simple statistical tests e.g., t-tests to analyse MEG data

But run lots of them - one at each time point/ sensor/ location/ frequency

Need to address the multiple comparisons problem

We can change the alpha level or use cluster correction

Also use ROIs, test a priori hypotheses, pre-register, etc.

Question 42

Which of the following statements about source estimation is FALSE?

A. Sensor space analyses does not require estimating a forward model

B. Timing of changes in brain activity can only be determined in sensor space

C. Source space analyses provide information about where in the brain activity is occuring

D. Inverse model is harder to compute than forward model

Answer 42

A - timing of changes in brain activity can be determined in both sensor and source space analyses

While sensor space analysis provides information from the sensors directly, source space analysis localizes brain activity and can also reveal temporal dynamics. Both methods can provide insights into the timing of brain activity changes, albeit from different perspectives.

Question 43

Why do we digitise the participant’s head shape before collecting MEG data?

A. To build a detailed 3D model of the participant’s head

B. To check that the head is the right size to fit in the MEG scanner

C. To calculate the location of the participant’s head in the helmet

D. To use in coregistering the data with structural MRI scans

Answer 43

D.

Question 44

The false discovery rate is:

A. The proportion of false positives across all tests

B. The proportion of true positives across all tests

C. The proportion of false positives across all significant tests

D. The proportion of true positives across all significant tests

Answer 44

C.

Question 45

In the cluster correction method considered in class, how is the largest cluster defined?

A. The chosen cluster has the largest absolute value of the summed test statistic

B. The chosen cluster has the longest duration and/or largest spatial extent

C. The chosen cluster contains the largest single value of the test statistic

D. The chosen cluster has the smallest summed p-value

Answer 45

A.