Week 3: GLM part 2 + Experimental designs

Question 1

What we can control in experimental designs

Answer 1

What we present and when

Question 2

Main effects

Answer 2

effects of a single condition, collapsing over the other(s). For example, testing whether red stimuli lead to different activity levels than green stimuli (regardless of shape) would be a test of a main effect

Question 3

Factorial design

Answer 3

Factorial designs are designs in which each event (e.g., stimulus) may be represented by a combination of different conditions. For example, you could show images of squares and circles (condition 1: shape) which may be either green or red (condition 2: color)

Question 4

In neuroscience, most hypotheses are…

Answer 4

directional (e.g., red > green)

Question 5

Parametric design

Answer 5

So far, we have discussed only designs with conditions that are categorical, such as “male vs. female faces” and “circles vs. squares”. The independent variables in your experimental design, however, do not have to be categorical! They can be continuous or ordinal, meaning that a particular variable might have different values (or “weights”) across trials. Designs involving continuously varying properties are often called parametric designs or parametric modulation. One hypothesis you might be interested in is whether there are voxels/brain regions which response is modulated by the reward magnitude (e.g., higher activity for larger rewards, or vice versa). In parametric designs, we create two regressors for every parametric modulation: one for the unmodulated response and one for the modulated response.

Question 6

To obtain large effects (i.e. t-values), we need three things

Answer 6

1. a large response/effect (e.g., beta)
2. an efficient design or, in other words, low design variance (can be done prior to collecting data)
3. Low noise/unexplained variance (this can be delat with during pre-processing)

Question 7

Efficiency is the inverse of…

Answer 7

the design variance (e.g., high eff = low design variance)

Question 8

Design variance is…

Answer 8

the part of the beta’s standard error that is caused by the design matrix (X)

Question 9

Researchers do not need to acquire (fMRI) data to calculate the efficiency of their design (X). Why?

Answer 9

We do not need to acquire data to calculate the efficiency because the formula for the efficiency only relies on the following values: stimuli, onsets and ordering of stimuli. In other words, the efficiency formula only relies on X (the design matrix), and not y (the actual signal)

Question 10

We want high variance within our predictors because…

Answer 10

…we want to base our model on a wide range of values that represent the whole population sample!

Question 11

Reason about this

Answer 11

You probably by now understand what’s the culprit: the design-variance! Given that the effect (beta IQ) is about the same for the two models and the MSE is higher for the high-variance model, the logical conclusion is that the design-variance of the high-variance model must be waaaaay lower.

Question 12

Detection

Answer 12

As you can see in the plot above, a blocked design groups trials of the same condition together in blocks, while the event-related design is completely random in the sequence of trials. Note that designs can of course also be a “mixture” between blocked and event-related (e.g., largely random with some “blocks” in between).

So, if we’re interested in detection (i.e., the amplitude of the response), what should we choose? Well, the answer is simple: blocked designs.

This is because blocked designs simply (almost always) have lower design variance because of:

lower covariance (“correlation”)
higher variance (“spread”)

Question 13

Block designs

Answer 13

• Bigger response between baseline and experimental > we are flooding stuff though, the nature of prediction is therefore weaker
• Similar events are grouped together
• Two condition block design with 16-20 sec blocks maximize power > but not always applicable!
• If we are interested in detection (which we almost always are), blocked designs are better because:
  > Lower covariance (no risk of overlap like in event designs)
  > Higher predictor variance = more “spread”

Question 14

Event-related designs

Answer 14

• Better!! (ignore what it says in the notebook!)
• Events are mixed together
• Good signal-to-noise ratio
• • jitter (semi-random ISI) = good for statistical efficiency (jitter = higher efficiency because we randomise the ISI)

Question 15

Psychological principles (1)

Stimulus predictability

Answer 15

• Influences psychological state
• e.g., go/no-go task: the predictability of the no-go stimulus determines how hard it is to not respond (event related better than block in this case)

Question 16

Psychological principles (2)

Time on task

Answer 16

• We can only image what subjects are doing, so they should be doing what we want them to do as much as possible
• Recognition time ~ 250 ms!! > remember that we recognise objects very fast, so the stimulus should be presented for <= 250 ms

Question 17

Psychological principles (3)

Participant strategy

Answer 17

• Different stimulus configurations afford different strategies (e.g., stroop task)
• Compatible trials vs. incompatible trials = compare these two to study cognitive control (e.g., people respond more slowly and less accurately in incompatible trials)
• If we take the stroop task, it makes more sense to use it with the blocked design

Question 18

Psychological principles (4)

Temporal precision of psychological manipulation

Answer 18

• What we expect from subjects should fit with what subjects can do
• e.g., sad vs. happy memories > block/event related is harder, people cannot switch between emotions that fast
• Solution: single epoch design (e.g., fixation baseline > emotional induction > emotional state > recovery induction > fixation baseline): this is viable because it does not present an emotion twice; emotions will not mean the same thing twice in a row !

Question 19

Psychological principles (5)

Unintended brain activity

Answer 19

• Brain imaging can capture all kinds of mental processes
• e.g., spatial attention shifting

Question 20

Overview

Experimental design

Answer 20

• Design: how many independent variables will be manipulated
• Trials: how are events organized (events, block, rapid, etc…)
• The design is all handled by the GLM! (BUT, there are other, novel ones that cannot be fully handled by the GLM: e.g., mediation, connectivity, classification/prediction, RSA)

Question 21

Kinds of designs (1)

Subtraction/pure insertion

Answer 21

• we can compare more complex conditions to simpler ones, by subtracting the activity of the simpler condition activity; what is left over must be due to the complex condition.
• the goal is to isolate a single neural process
• assumes that neural processes can be summed linearly
• assumes that the neural processes associted to each task are NOT interacting with eachother

Issues: interaction with context & pure insertion often violated

Question 22

Kinds of designs (2)

Multiple subtraction

Answer 22

• e.g., (task A – task B – task C)
• Can avoid issues with pure insertion (context issue)
• are useful for increasing specificity of the conclusions you can draw from your results.

Question 23

Kinds of designs (3)

Factorial

Answer 23

• an approach that characterizes interaction between processes
• interaction effect = main effect 1 x main effect 2 (e.g., gender x expression)

Question 24

Kinds of designs (4)

Parametric

Answer 24

• Designs involving continuously varying properties (different parametric values) > continuous independent variables (vs. discrete as in factorial)
• Key takeaway: Does the BOLD signal seem to increase as the intensity of the stimulus increase? > increase our confidence that there is a relationship
• Two levels (we assume that our design affects the voxel response in two ways):
  > Unmodulated response > a response to the task independent of the parametric value
  > PREDICTOR: stick predictor (1s and 0s)
  > Modulated response > a response to the task dependent on the parametric value
  > PREDICTOR: mean-subtracted e.g., reward magnitude; we subtract to decorrelate modulated from unmodulated responses

Very likely, the betas for the modulated predictor will have a greater effect on the voxel activity than the unmodulated predictor

Example: One hypothesis you might be interested in is whether there are voxels/brain regions whose response is modulated by the reward magnitude (e.g., higher activity for larger rewards, or vice versa)

Question 25

IMPORTANT trade-off

Answer 25

• Only one comparison/condition
  > More power, but less generalizability> Stick with this one at the beginning of studying a new field
• Many comparisons/conditions
  Low power, but more generalizability

Question 26

Estimation vs. detection

Answer 26

Detection: you want to know whether different conditions activate voxel activity differently > Is there signal? > magnitude / amplitude!

Estimation: You want to investigate how different conditions influence voxel activity not only by investigating the “amplitude” parameter of the HRF, but also parameters relating to other properties of the shape of the HRF (like width, lag, strength of undershoot, etc.). > What is the shape of the fMRI signal?

Question 27

Detection properties

Answer 27

• Better with blocked designs
• Estimates only ONE beta parameters for the stimulus regressor (reflecting activation/deactivation)
• Usually, employes canonical HRF convolution

Question 28

Estimation properties

Answer 28

• Better with event related designs
• By employing FIR models, it estimates betas for each set time point for the stimulus regressor (reflecting activation/deactivation)
• Usually employes FIR (finite impulse response) models

Question 29

FIR models

Answer 29

• finite impulse response model
• less biased than canonical HRF, but more variable
• makes no assumptions about the shape of the HRF
• esimtaes a beta for each set time point: more computationally expensive, if our prior assumption about shape is correct, then it is very efficient, otherwise very inefficient

Question 30

Constrained basis set

Answer 30

• Can contain any number of parts / functions
• Lower the number of regressors, BUT…
• …constrained to the shape that are reasonable for the HRF shapes
• so it is like an in-between of the canonical HRF and the FIR models

Question 31

Advanatges/Disadvantages of canonical HRF

Answer 31

ADV:
* simple analysis
* one parameter per predictor
* easily interpretable
* simple

DIS:
* strong assumptions about shape
* biased if it estimates the shape of the BOLD incorrectly

Question 32

Advanatges/Disadvantages of FIR ( = unbiased basis set)

Answer 32

ADV:
* not biased towards specific shape, weak assumptions about shape
* allows testing of HYP about specific HRF parameters
* can model the width, height and time delay of the BOLD response quite well

DIS:
* less powerful ( because we have more parameters (?) )
* makes group analysis more difficult (because every subject would have sooo many more betas?)

Question 33

Limitations of BOLD linearity

Answer 33

• BOLD response towards short stimuli stronger than predicted
• Apparently not possible to deliver only ‘some’ oxygen
• Stimuli shown for less than 1-2 seconds behave
  as lasting 1-2 seconds
• BOLD response saturates at particular moment
• Longer duration of stimulation does not further add to the response
• Modeled by including a maximum response

Question 34

Bias/Varinace equal…

Answer 34

Validity/Reliability

Question 35

Validity

Answer 35

internal/external/construct
* is the observed effect due to our manipulation?
* do our results generalize?
* are we measuring what we want to measure?

Question 36

Reliability

Answer 36

How robust is the data? (task, design, contrast, thresholding can all affect reliability)

Question 37

Efficiency

Answer 37

The chunk of variance we can look at

Question 38

T-statistic

Answer 38

We use it to evaluate whether a beta parameter differs significantly from 0 / whether a voxel activates/deactivates significantly in response to one or more experimental factors

Question 39

Uncertainty

Answer 39

given the parameters we want to test, we need to test them in the context of their uncertainty

Question 40

The raw value of the beta parameter(s) and the MSE are dependent on the…

Answer 40

…scale of the variables (we can rescale either X, or y!)

Therefore, how can we interpret the effects of our predictors?

Question 41

How can we get interpretable parameters?

Answer 41

By dividing the raw betas by a value that quantifies how well our model describes the data

The effect (beta_hat) divided by our uncertainty in the effect (SE)

Question 42

If the effect is large, but the uncertainty is also large (bad model fit), then we get a [low/high] t-value

Answer 42

low

Question 43

If the effect is large, and the uncertainty low (good model fit), then we get a [low/high] t-value

Answer 43

high

Question 44

Standard error (SE) of the model comprises two components

Answer 44

Unexplained variance of the model (noise) * the design variance (the variance of the parameter due to the design)

Question 45

The SE(beta_hat) equals the…

Answer 45

var(beta_hat)

Question 46

SE(beta_hat) = … (formula)

Answer 46

sqrt(noise * design variance)

They scale eachother, both should be small for a high t-value

Question 47

Design variance can be attenuated by controlling the…; noise can be attenuated by controlling the … .

Answer 47

experimental design;preprocessing

Question 48

Noise formula (sigma^2) and predictor variance formula (var(x))

Answer 48

1. sum of squared errors / df
2. sum of squared difference between each x sample and the mean of the x values / df

Question 49

Efficiency

Answer 49

inverse of design variance; 1/design variance

Question 50

What causes a low design variance (and hence high efficiency in desing?)

Answer 50

1. predictors should have high variance (they should vary a lot relative to their mean)
2. predictors should not have high covariance (they should not correlate between each other a lot)

Question 51

Extended formula for efficiency

Answer 51

var(Xj) + var(Xk) -2 * cov(Xj, Xk)

We want the sum of the variances to be high compared to the scaled covar

Question 52

T/F: the absolute value of efficiency is not interpretable, but we can interpret it relative to other designs

Answer 52

True

Question 53

The matrix version for the efficiency formula

Answer 53

c(X.T * X) ^(-1) * c.T

works for both testing betas against eachother and one beta vs. baseline

Question 54

The effect of predictor variance on the design variance/efficiency

Answer 54

The more values of a variable deviate from its mean on average, the more variance the variable has

Question 55

The effect of predictor covariance on the design variance/efficiency

Answer 55

• The higher the covariance, the lower the design efficiency
• High covariance between predictors causes uncertainty of the estimation of our beta-estimates
• If the predictors are correlated, the GLM “doesn’t know” what (de)activation it should assign to which predictor; this causes a higher design variance term

Question 56

P-value on t-stat

Answer 56

area under the curve of a t-distribution associated with our t-value and more extreme values (week 2)

Question 57

Types of t-tests

Answer 57

• Using contrast between parameter and baseline

Does my predictor have any effect on my dependent variable?

• Using a contrast between parameters

Does predictor 1 have a larger effect on my dependent variable than predictor 2?

• Why do we use contrasts?

Not only because they are “elegant”, but also because they allow us to specify the exact hypothesis, we are interested in

Question 58

F-tests

Answer 58

Used to calculate a significance value for multiple contrasts

Example: “Does the presentation of any circle (regardless of colour) activate a voxel compared to baseline?”

Question 59

How can we maximize efficiency?

Answer 59

1. Come up with different design matrices
2. Make sure we are maximising for the correct efficiency; if we care about HRF shape, we want to maximise efficiency for ESTIMAITON!

Question 60

I didn’t know this about contrasts (but it makes sense)

Answer 60

When we are comparing between conditions, the 1s or any type of value in the contrast should sum to 0, but not when we are comparing to baseline.

Example: [0, -1, -1, -1] is interpreted as: are the voxels on average negative (and the opposite would apply if all the 1s were positive)

Question 61

ISI (interstimlus interval)

Answer 61

• Usually, researchers vary the ISI from trial to trial > jittering
• Drawing ISIs randomly from a normal distribution > this may yield more efficient designs by reducing covariance and increasing predictor variance
• NEVERTHELESS, jittering does not always improve design efficiency, but by injecting randomness, it allows for a larger variety of designs, which also include designs that happen to be more efficient than the fixed-ISI designs.

Question 62

Double dissociation vs. Separate modifiability

Answer 62

DD assumes that two tasks (A and B) both activate two areas, but task A activates brain area1 more than B while B activates brain area2 more than A: SM states that A (but not B) activates brain area1 and vice versa for brain area2.

Question 63

Connection between simple subtraction and pure insertion

Answer 63

Applying the subtraction method assumes that pure insertion holds: that is, a mental process can be inserted into a stream of processes without affecting other processes