Stats Validity

Question 1

Internal validity

Answer 1

Internal validity is the confidence that we can place in the cause and effect relationship in a study. It is the confidence that we have that the change in the independent variable caused the observed change in the dependent variable (rather than due to poor control of extraneous variables)

Question 2

Threats to internal validity

Answer 2

Reliability of measurement instruments
Regression towards the mean (subjects selected based on extreme scores will tend to regress spontaneously towards the mean on subsequent tests)
Sampling
Experimental mortality (loss of participants over time may result in unequal characteristics in two groups)
Instrument obtrusiveness (the instrument should not affect the data collection e.g. poorly designed questionnaires)
Manipulation effectiveness (the independent variable must be manipulated enough so that the effect can be seen, ideally the degree of manipulation should be measured)
History (where two measurements of the dependent variable occur that are separated in time, there is the potential for various other influences to get introduced)
Maturation (people mature over time and this may in itself explain the change of a dependent variable)
Measurement sensitisation (the instrument may affect the way the subject see’s the world and so may bias future measures)
Measurement instrument learning (people may get used to the measurement instrument, a good example is the increasing performance on repeated use of the WAIS for estimation of IQ)

Question 3

External validity

Answer 3

External validity is the degree to which the conclusions in a study would hold for other persons in other places and at other times, i.e. its ability to generalise.

Question 4

Threats to external validity

Answer 4

Representativeness of the sample
Reactive effects of setting (is the research setting artificial)
Effect of testing (if a pre-test was used in the study that will not be used in the real world this may affect outcomes)
Multiple treatment inference (this refers to study’s in which subject receive more than one treatment, the effects of multiple treatments may interact)

Question 5

Face validity

Answer 5

Face validity refers to the general impression of a test. A test has face validity if it appears to test what it is meant to

Question 6

Content validity

Answer 6

Content validity refers to the extent to which a test or measure assesses the full content of a subject or area. For example if a test is designed to help diagnose depression, it would have poor content validity if it only asked about psychological symptoms and neglected biological ones

Question 7

Criterion validity

Answer 7

Criterion validity concerns the comparison of tests. You may wish to compare a new test to see if it works as well as an old, accepted method. The correlation coefficient is used to test such comparisons

Question 8

Criterion validity (Concurrent)*

Answer 8

In concurrent validation, the predictor and criterion data are collected at or about the same time. An example could be testing a new, shorter test of intellectual functioning against a standard measure

Question 9

Criterion validity (Predictive)*

Answer 9

In Predictive validation, the predictor scores are collected first and criterion data are collected at some later/future point. Here you want to know if the test predicts future outcomes. An example might be evaluating a new assessment method to select medical students. The test could be compared against the students performance at the end of year one to see if there is a correlation

Question 10

Construct validity

Answer 10

The extent to which a test measures the construct it aims to

Question 11

Construct validity (Convergent)*

Answer 11

A test has convergent validity if it has a high correlation with another test that measures the same construct

Question 12

Construct validity (Divergent)*

Answer 12

A test’s divergent validity is demonstrated through a low correlation with a test that measures a different construct