Brainscape's Knowledge GenomeTM


Top Flashcards (Certified)
All Flashcards

MRCPsych Part B > Reliability & Validity > Flashcards

Reliability & Validity Flashcards

1
Q

What can high reliability guarantee?

A

Consistency

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

How can one test reliability?

A

Test-retest correlation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

How does test-retest correlation work?

A

Administering an instrument twice to same population

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

How does one avoid the practice effect when doing test-retest?

A

Time difference must be long enough
But short enough so the underlying state does not change

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Time difference for test-retest in psychiatric studies

A

2-14 days

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What measures internal consistency of a test?

A

Cronbachs alpha

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

How does cronbachs alpha test internal consistency?

A

By correlating each item with the total score and averaging the correlation coefficient

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Values of Cronbachs alpha

A

Negative infinity to 1

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What Cronbachs alpha values make sense

A

Positive values only

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Cut off for Cronbachs alpha for a test to be internally consistent?

A

0.7

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

What is split half reliability?

A

Splitting scale in two parts and examining the correlation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What is intraclass correlation coefficient used for?

A

Continuous variables

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What is the intraclass correlation coefficient?

A

Proportion of total variables of measurement that reflects true between subject variability

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Range of intraclass correlation coefficient?

A

0 (unreliable) - 1 (perfect reliability)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What can ICC be measured for?

A

Relative or absolute agreement

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Difference between relative and absolute agreement

A

Relative ICC is always higher

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Levels of ICC and their meanings

A

0.6 = fair
0.8 = very good
0.9 = excellent

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What is ANOVA intraclass coefficient used for?

A

Quantitative data with more than 2 rates/groups

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

What is used to test relaibility for nominal data with more than 2 categories?

A

Kappa or weighted kappa

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What is face validity?

A

Subjective measure of deciding whether a test measures the construct of interest at face value

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Types of construct validity

A

Content
Criterion
Convergent
Discriminant
Experimental

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

What is criterion validity made up of

A

Concurrent
Predictive

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

What is construct validity?

A

Measures whether a test really measures the construct of interest

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

What is unified construct validity?

A

Both content and criterion validity

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
What is content validity?
Whether the contents of the test are in line with specifications the test was designed to measure
26
What does content validity look for?
Good coverage of all domains thought to be related to the measured condition
27
How does one measure content validity?
Cannot be statistically tested Experts are called to comment on this validity
28
What is criterion validity?
Performance of a test against an external criterion such as an instrument or future diagnstic possibility
29
What is concurrent validity?
Ability of a test to distinguish between subjects who differ concurrently in other measures (using other instruments)
30
What is predictive validity?
Ability of a test to predict future group differences according to current group scores
31
What is incremental validity?
Ability of a measure to predict or explain variance over and above other measures
32
What can one divide construct validity into?
Concurrent & predictive Convergent, discriminant & experimental Factorial
33
What is convergent validity?
Agreement between instruments that measure same construct
34
What is discriminant validity?
Degree of disagreement between two scales measuring different constructs
35
What is experimental validity?
Sensitivity to change.
36
What is factorial validity?
Established via factor analysis of items in a scale
37
What is precision?
Degree to which the mean varies with repeated sampling
38
What leads to imprecision?
Random errors
39
Factors that reduce precision
Wide interval limits Expecting higher CI
40
What is accuracy?
Correctness of the mean value i.e. how close it is to the true population value
41
What compromises both validity and accuracy?
Bias
42
Disadvantages of percent agreement?
Overestimates degree of agreement
43
What does kappa indicate?
Level of agreement that could be expected beyond chance
44
What is kappa used for?
Agreement on categorical variables
45
What is weighted kappa used for?
Ordinal variables
46
What is used for beyond chance agreement in continuous variables?
Bland-Altman plot
47
Degree of agreement if kappa is 0
None
48
Degree of agreement if kappa is 0-0.2
Slight
49
Degree of agreement if kappa is 0.2-0.4
Fair
50
Degree of agreement if kappa is 0.4-0.6
Moderate
51
Degree of agreement if kappa is 0.6-0.8
Substantial
52
Degree of agreement if kappa is 0.8-1.0
Almost perfect
53
What affects kappa?
Prevalence of outcome studied - higher proportion of assessments leads to higher kappa
54
Calculations for kappa
(observed agreement beyond chance) / (maximum agreement beyond chance) OR (observed agreement - agreement by chance) / (100% - agreement expected by chance)
55
What numerical values are needed to calculate kappa?
Percentage of patients that the 2 assessors correctly classified Expected agreement be chance
56
What is kappa dependent on?
Prevalence of measured condition
57
What type of disorders will kappa be low for?
Common disorders
58
Disadvantage of kappa
One cannot test statistical significance from kappa values
59
What is another way of calculating beyond chance agreement for nominal values
Phi
60
Advantages of phi
Statistical significance testing is possible Small sample size can be used
61
What is plotted in bland-altman plot?
Pairs of score differences are plotted against the mean

MRCPsych Part B (64 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2025 Bold Learning Solutions. Terms and Conditions