Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

MRCPsych Part B > Reliability & Validity > Flashcards

Reliability & Validity Flashcards

1
Q

What can high reliability guarantee?

A

Consistency

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

How can one test reliability?

A

Test-retest correlation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

How does test-retest correlation work?

A

Administering an instrument twice to same population

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

How does one avoid the practice effect when doing test-retest?

A

Time difference must be long enough

But short enough so the underlying state does not change

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Time difference for test-retest in psychiatric studies

A

2-14 days

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What measures internal consistency of a test?

A

Cronbachs alpha

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

How does cronbachs alpha test internal consistency?

A

By correlating each item with the total score and averaging the correlation coefficient

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Values of Cronbachs alpha

A

Negative infinity to 1

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What Cronbachs alpha values make sense

A

Positive values only

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Cut off for Cronbachs alpha for a test to be internally consistent?

A

0.7

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

What is split half reliability?

A

Splitting scale in two parts and examining the correlation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What is intraclass correlation coefficient used for?

A

Continuous variables

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What is the intraclass correlation coefficient?

A

Proportion of total variables of measurement that reflects true between subject variability

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Range of intraclass correlation coefficient?

A

0 (unreliable) - 1 (perfect reliability)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What can ICC be measured for?

A

Relative or absolute agreement

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Difference between relative and absolute agreement

A

Relative ICC is always higher

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Levels of ICC and their meanings

A
  1. 6 = fair
  2. 8 = very good
  3. 9 = excellent
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What is ANOVA intraclass coefficient used for?

A

Quantitative data with more than 2 rates/groups

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

What is used to test relaibility for nominal data with more than 2 categories?

A

Kappa or weighted kappa

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What is face validity?

A

Subjective measure of deciding whether a test measures the construct of interest at face value

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Types of construct validity

A 
Content
Criterion
Convergent
Discriminant
Experimental
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

What is criterion validity made up of

A

Concurrent

Predictive

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

What is construct validity?

A

Measures whether a test really measures the construct of interest

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

What is unified construct validity?

A

Both content and criterion validity

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

What is content validity?

A

Whether the contents of the test are in line with specifications the test was designed to measure

26
Q

What does content validity look for?

A

Good coverage of all domains thought to be related to the measured condition

27
Q

How does one measure content validity?

A

Cannot be statistically tested

Experts are called to comment on this validity

28
Q

What is criterion validity?

A

Performance of a test against an external criterion such as an instrument or future diagnstic possibility

29
Q

What is concurrent validity?

A

Ability of a test to distinguish between subjects who differ concurrently in other measures (using other instruments)

30
Q

What is predictive validity?

A

Ability of a test to predict future group differences according to current group scores

31
Q

What is incremental validity?

A

Ability of a measure to predict or explain variance over and above other measures

32
Q

What can one divide construct validity into?

A

Concurrent & predictive
Convergent, discriminant & experimental
Factorial

33
Q

What is convergent validity?

A

Agreement between instruments that measure same construct

34
Q

What is discriminant validity?

A

Degree of disagreement between two scales measuring different constructs

35
Q

What is experimental validity?

A

Sensitivity to change.

36
Q

What is factorial validity?

A

Established via factor analysis of items in a scale

37
Q

What is precision?

A

Degree to which the mean varies with repeated sampling

38
Q

What leads to imprecision?

A

Random errors

39
Q

Factors that reduce precision

A

Wide interval limits

Expecting higher CI

40
Q

What is accuracy?

A

Correctness of the mean value i.e. how close it is to the true population value

41
Q

What compromises both validity and accuracy?

A

Bias

42
Q

Disadvantages of percent agreement?

A

Overestimates degree of agreement

43
Q

What does kappa indicate?

A

Level of agreement that could be expected beyond chance

44
Q

What is kappa used for?

A

Agreement on categorical variables

45
Q

What is weighted kappa used for?

A

Ordinal variables

46
Q

What is used for beyond chance agreement in continuous variables?

A

Bland-Altman plot

47
Q

Degree of agreement if kappa is 0

A

None

48
Q

Degree of agreement if kappa is 0-0.2

A

Slight

49
Q

Degree of agreement if kappa is 0.2-0.4

A

Fair

50
Q

Degree of agreement if kappa is 0.4-0.6

A

Moderate

51
Q

Degree of agreement if kappa is 0.6-0.8

A

Substantial

52
Q

Degree of agreement if kappa is 0.8-1.0

A

Almost perfect

53
Q

What affects kappa?

A

Prevalence of outcome studied - higher proportion of assessments leads to higher kappa

54
Q

Calculations for kappa

A

(observed agreement beyond chance) / (maximum agreement beyond chance)

OR

(observed agreement - agreement by chance) / (100% - agreement expected by chance)

55
Q

What numerical values are needed to calculate kappa?

A

Percentage of patients that the 2 assessors correctly classified
Expected agreement be chance

56
Q

What is kappa dependent on?

A

Prevalence of measured condition

57
Q

What type of disorders will kappa be low for?

A

Common disorders

58
Q

Disadvantage of kappa

A

One cannot test statistical significance from kappa values

59
Q

What is another way of calculating beyond chance agreement for nominal values

A

Phi

60
Q

Advantages of phi

A

Statistical significance testing is possible

Small sample size can be used

61
Q

What is plotted in bland-altman plot?

A

Pairs of score differences are plotted against the mean

MRCPsych Part B (64 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions