Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Advanced statistics > Reliability and Validity > Flashcards

Reliability and Validity Flashcards

1
Q

Reliability

A
  • refers to the replicable nature of research studies/tools

- high reliability does not guarantee scientific validity

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Testing reliability

A
  • test-retest correlation
  • instrument is twice administered to the same population
  • 2-14 days in between
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Cronbach’s alpha

A
  • measures internal consistency of a test by correlating each item with the total score and averaging the correlation coefficients
  • it takes values between negative intinity and 1 as maximum but only positive values make sense
  • arbitrary cut-off of 0.70 is used to call the evaluated test to be internally consisten
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Split-half reliability

A

-refers to splitting a scale into two parts and examining the correlation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Interrater reliability

A

-measured using two or more raters rating the same population using the same scale

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Intraclass correlation coefficient

A
  • used for continuous variables
  • the proportion of total variance of the measurement that reflects true between subject variability
  • ranges between 0 (unreliable) and 1 (perfect reliability)
  • relative ICC is always higher than absolute ICC
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

ANOVA intraclass coefficient

A

-used for quantitative data of more than 2 raters/groups

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Nominal data reliability

A

-if it has more than two categories then a kappa or weighted kappa can be used

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Validity of an instrument

A

-the extent to which an instrument measures what it proposes to measure

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Face validity

A

-refers to a subjective measure of deciding whether the test measures the construct of interest on its face value (what it was designed for)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Construct validity

A

-measures whether a test really measures the theoretical construct of interest of something else

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Content validity

A

-refers to whether the contents ie each individual subscales, items or elements of the test are in line with the general objectives or specifications the test was originally designed to measure

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Criterion validity

A

-refers to performance against an external criterion such as another instrument (concurrent) or future diagnostic possibility (predictive)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Concurrent validity

A

-refers to the ability of a test to distinguish between subjects who differ concurrently in other measures e.g those who score high on a scale of insomnia may score high on the scale of fatigue ratings

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Predictive validity

A

-the ability of a test to predict future group differences according to current group differences in score

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Incremental validity

A

-refers to the ability of a measure to predict or explain variance over and above other measures

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Convergent validity

A
  • refers to the agreement between instruments that measure the same construct
  • form of construct validity
18
Q

Discriminant validity

A
  • refers to the degree of disagreement between two scales measuring different constructs
  • form of construct validity
19
Q

Experimental validity

A
  • refers to the sensitivity to change
  • an instrument must show the difference in results when an intervention is carried out to modify the measured domain
  • form of construct validity
20
Q

Factorial validity

A

-form of construct validity established via factor analysis of items in a scale

21
Q

Precision

A
  • degree to which a calculated central value varies with repeated sampling
  • the more narrow the variation the more precise the value is
  • random error leads to imprecision
22
Q

Factors reducing precision

A
  • wider limits of the interval

- expecting higher confidence interval

23
Q

Accuracy

A

-the correctness of the mean value

24
Q

Precision

A

-comparable to reliability while accuracy is comparable to validity

25
Q

Bias

A

-compromises validity/accuracy

26
Q

Face

A

-does the scale appear to be fit for the purpose of measuring the variable of interest?

27
Q

Content

A

-does the scale appear to include all the important domains of the measured attribute?

28
Q

Criterion

A

-is the scale consistent with what we already know (concurrent) and what we expect (predictive)?

29
Q

Convergent

A

-does this new scale associate with a different scale that measures a similar construct?

30
Q

Discriminant

A

-does the new scale disagree with scales that measure unrelated constructs?

31
Q

Kappa

A

observed agreement beyond chance/maximum agreement beyond chance
OR
(observed agreement-agreement expected by chance)/(100%-agreement expected by chance)

32
Q

Beyond chance agreement

A

-Kappa indicates the level of agreement that could be expected beyond chance

33
Q

When is Kappa calculated?

A

-only for agreement on categorical variables such as presence or absence of a diagnosis

34
Q

Weighted Kappa

A

-for ordinal variables

35
Q

Bland-Altman

A

-used for continuous variables where pairs of score differences are plotted against the mean

36
Q

Kappa values and degree of agreement

A 
0= no agreement
0-0.2=slight
0.2-0.4=fair
0.4-0.6= moderate
0.6-0.8=substantial
0.8-1.0= almost perfect
37
Q

What is kappa statistics?

A
  • if two investigators independently assess the same group, there will be an extent to which their results ‘agree’
  • simple percent agreement overestimates the degree of agreement and this is misleading. This is why kappa statistics are done
  • kappa indicates the level of agreement that could be expected beyond chance
38
Q

Interpreting kappa

A
  • affected by the prevalence of the outcome studied -the higher the proportion of positive assessments the higher the kappa
  • statistical significance cannot be tested directly from kappa measurements but it guides the actual degree of agreement
39
Q

Observed agreement calculation

A
  • draw two by two table
  • take the two values agreed and add them together
  • then divide by 100
40
Q

Kappa coefficient

A
  • dependent on the prevalence of the measured condition
  • common disorders will have low kappa but rare disorders have low Kappa
  • one must look at the actual percentage afreement
41
Q

Phi

A
  • similar to kappa
  • all cells are utilised and statistical significance is possible
  • small sample size can be used

Advanced statistics (16 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions