Lecture 9 Flashcards

1
Q

What is a type I error?

A

(alpha - related to significance level) When Ho is true but it is rejected, giving a false positive.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What is a type II error?

A

(Beta) when Ho is false but it is accepted, giving a false negative.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

How can we control the number of type 1 errors?

A

By controlling out significance level alpha!

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What are the two main groups of methods for accounting for multiple tests?

A

Adjustment of p-values and determining p-values

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What is involved with adjustment of p-values?

A

Controlling family wise type 1 error rates (FWER)

Controlling false discovery rate (FDR)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What is involved with determination of p-values?

A

Permutation test

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Define Per-comparison error rate in equation form.

A

PCER=E(V)/m

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Define family-wise error rate in equation form.

A

FWER=P(V≥1)= 1-P(V=0)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Define False discovery rate in equation form.

A

FDR=E(V/R|R>0)*P(R>0)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Define Proportion of false positives

A

PFP=E(V)/E(R)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Which ‘rate’ is related to all tests?

A

FWER

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What ‘rate’ is related to only rejected hypotheses?

A

FDR

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Define single step procedures.

A

Equivalent adjustments are performed for all hypotheses

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What are examples of single step procedures?

A

Bonferroni and sidak adjustments

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Define stepwise procedures

A

Adjustments based not only on m but also on outcome of all the tests

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Give examples of stepwise procedures

A

Benjamin and Hochberg adjustment

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

What procedures/methods control FWER?

A

single step procedures - bonferroni and sidak

18
Q

What procedures/methods control FDR?

A

stepwise procedures - benjamin and hochberg

19
Q

Explain the bonferroni correction.

A

Rejects any null hypothesis Hj with p-values less than or equal to alpha/m.

20
Q

What are some characteristics of the bonferroni correction?

A

Strong control of FWER at level alpha
Suitable for situations where no type I error is tolerated
Not well suited when several Ho are not true (or several discoveries are expected
Low power for detection

21
Q

Finish this statement: The more you control type I error….

A

The less power you will have.

22
Q

Explain the Sidak correction

A

Rejects any hypothesis Hj with p-value less than or equal to 1-(1-alpha)^(l/m)

23
Q

What are some characteristics of the Sidak correction?

A

Very similar to the bonferroni adjustment
Both are too conservative for our application(mapping)
These methods do not take into account dependence between tests (linked markers or correlated traits)

24
Q

What is a disadvantage to the bonferroni and sidak corrections?

A

These methods do not take into account dependence between tests (linked markers or correlated traits)

25
Q

What are methods that control FDR based on?

A

Number of rejections, not only number of tests

26
Q

What are methods that control FWER based on?

A

number of tests

27
Q

What is the general characteristics of FDR controlling methods? What does that mean for this course?

A

They provide weak control of FWER but have a higher power, while controlling FDR. This means we can use them for mapping!

28
Q

What did benjamin and hochberg prove in 1994?

A

That the FDR can be controlled at some level q, by determining the largest i for which: q

29
Q

What does the benjamin and hochberg adjustment assume?

A

That tests are independent

30
Q

Explain how the Benjamin and Hochberg adjustment works.

A

Consider testing m null hypotheses Hi (i=l,…,m) and the corresponding computed p-values P1, P2, …,Pm ordered in ascending order. Denote Hi the null hypothesis corresponding to Pi.

31
Q

Who proposed permutation tests? what was the method for?

A

Churchill and Doerge in 1994

Proposed a method to empirically estimate FWER rejection thresholds

32
Q

What is churchill and Doerge’s method based on?

A

Permutation tests for simulated data

33
Q

What assumptions does Churchill & Doerge/Permutation tests make with respect to the distribution of tests statistics under the null hypothesis?

A

It does not make any assumptions, the distribution itself is not important

34
Q

What are the three steps of the permutation test algorithm?

A

1: Randomly Shuffles the observed phenotypes over individuals (marker genotypes)
2: Repeat #1 many times (resampling)
3: Evaluate the empirical distribution of the test statistics under the null hypothesis generate above to determine the threshold levels for CWER and FWER

35
Q

Explain the first step of the permutation test algorithm in detail. (like a short answer)

A

Randomly shuffles the observed phenotypes over individuals (marker genotypes). This is a sample of original marker genotypes but with the phenotypic values randomly assigned, which will provide a sample of the test statistics under the null hypothesis of no-marker trait association. This breaks the association between markers and phenotypes (there is no difference between means of marker alleles).

36
Q

What are the three steps of a permutation test algorithm for models with polygenic effect?

A

1: Randomly Shuffles the observed GENOTYPES over individuals (keep intact relationship phenotype-polygenes). This is a sample of original phenotypic values but with the genotypes randomly assigned.
2: Repeat #1 many times (resampling)
3: Evaluate the empirical distribution of the test statistics under the null hypothesis generate above to determine the threshold levels for CWER and FWER

37
Q

What is the difference between a normal permutation test algorithm and one for models with polygenic effects?

A

Normal:
-Shuffles phenotypes over marker genotypes
-Sample is original marker genotypes with phenotypic values randomly assigned
Polygenic:
-Shuffles Genotypes over individuals, keeping intact relationship phenotype-polygenes
-Sample is original phenotypic values with genotypes randomly assigned

38
Q

What number of resamplings are suggested by Churchill and Doerge to be sufficient at the 5% and 1% significance levels, respectively?

A

1,000 resamplings for 5%

10,000 may be needed for 1%

39
Q

What does permutation tests account for?

A

Missing markers and differences in density of markers

40
Q

What is a disadvantage to the permutation test?

A

it is very time consuming