Week 8: Interpretation of test results

Question 1

A person’s raw score has little meaning without which two things?

Answer 1

1. A comparison to a normative sample
2. A method for interpreting the meaning of the comparison

Question 2

What do we call a measure that can be used to compare values from different data sets?

Answer 2

Relative standing

Question 3

What is it called when interpreting and communicating test performance depends on having an appropriate comparative sample and a common ‘language’ of descriptions?

Answer 3

Rule of thumb

Question 4

What is a positively skewed distribution?

Answer 4

More scores fall below the mean compared to above the mean (left side higher and right side lower)

Question 5

What is a negatively skewed distribution?

Answer 5

More scores fall above the mean compared to below the mean (left side lower and right side higher)

Question 6

What happens when scores from a normative sample are not normally distributed (2)?

Answer 6

1. The mean and median are not identical
2. Z-scores will not accurately translate into sample percentile rank values

Question 7

A … sample size will produce a … normal distribution, but only if the underlying characteristic in the population distribution obtained is normal

Answer 7

Larger, more

Question 8

When can a truncated distribution (where the starting point is not 0) occur (2)?

Answer 8

1. When scores are restricted at one side of the distribution
2. When specific subgroups are purposefully excluded from inclusion in the normative sample

Question 9

A truncated distribution of scores can lead to (3)?

Answer 9

1. Identification of normal individuals as low functioning
2. Difficulty estimating the severity of impaired performance
3. An increase in number of persons identified as impaired

Question 10

When is it useful to compare scores between tests (2)?

Answer 10

1. The raw score distributions for tests that are being compared are approximately normal in the population
2. The scores that are being compared are derived from similar samples

Question 11

When comparing test scores, it is important to consider the … of two measures and their …

Answer 11

Reliability, intercorrelation

Question 12

The relationship between normative scores and percentiles are lineair/non-lineair

Answer 12

Non-lineair

Question 13

What is defined as the presence of truncated tails in the context of limitations in range of item difficulty?

Answer 13

Ceiling and floor effects

Question 14

What does a high floor in scores mean?

Answer 14

When a large proportion of the examinees obtain raw scores at or near the lowest possible score

Question 15

What indicates a high floor in test scores?

Answer 15

That the test lacks a sufficient number and range of easier items

Question 16

Floor and ceiling effects can lead to?

Answer 16

Misinterpretations results

Question 17

What does extrapolation entail?

Answer 17

The action of estimating or concluding something by assuming that existing trends will continue. When norms fall short in terms of rang this technique is often used

Question 18

Comparison of performance across tests is affected by: (5)

Answer 18

1. Measurement error
2. Score magnitude
3. Extreme scores
4. Ceiling and floor effects
5. Extrapolation/ interpolation of derived scores

Question 19

It is important to carefully consider how to interpret isolated low scores. The likelihood of obtaining low scores increases when (3)?

Answer 19

1. The number of tests increases
2. The cut off for defining low scores becomes more open-minded
3. With lower levels of baseline cognitive functioning

Question 20

The degree of agreement between different people that are observing or assessing the same thing = (Inter-rater reliability/Test-retest reliability/Parallel-forms reliability/Internal consistency reliability)

Answer 20

Inter-rater reliability

Question 21

Measure the consistency of the result when you repeat the measure the same thing at a different point of time = (Inter-rater reliability/Test-retest reliability/Parallel-forms reliability/Internal consistency reliability)

Answer 21

Test-retest reliability

Question 22

Measures the correlation between two equivalent versions of a test. This can help to avoid practice effects, but the versions should be equivalent = (Inter-rater reliability/Test-retest reliability/Parallel-forms reliability/Internal consistency reliability)

Answer 22

Parallel-forms reliability

Question 23

The correlation between items
within a test that are mean to measure the same construct = (Inter-rater reliability/Test-retest reliability/Parallel-forms reliability/Internal consistency reliability)

Answer 23

Internal consistency reliability

Question 24

What is validity?

Answer 24

Validity is the degree to which a test is measuring what is was intended to measure

Question 25

What is reliability?

Answer 25

The consistency of a measure (whether the results can be reproduced under the same conditions)

Question 26

What is sensitivity?

Answer 26

Sensitivity is the probability of a positive test, given that the person is affected

Question 27

What is specitifity?

Answer 27

The probability of a negative test, given that a person is healthy

Question 28

What does a p-value NOT measure (3)?

Answer 28

1. Does not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone
2. They do not provide a good measure of evidence regarding a model of hypothesis
3. They do not measure the size of an effect or the importance of a result

Question 29

What does a p-value measure?

Answer 29

The probability that a statistical summary of the data would be as extreme (or more) than its observed value

Question 30

What is circular analysis?

Answer 30

Circular analysis is any form of analysis that retrospectively selects features of the data to characterise the dependent variables, resulting in a distortion of the resulting statistical test
= based on data that was selected for showing the effect of interest or a related effect

Question 31

What is p-hacking?

Answer 31

The misreporting of true effect sizes in published studies. It occurs when researchers try out several statistical analyses and then selectively report those that produce significant results

Question 32

What is a spurious correlation?

Answer 32

Occurs when two factors appear casually related to one another but are not. Spurious correlations most commonly arise if one or several outliers are present for one of the two variables

Question 33

Is the test fully representative of what it aims to measure, refers to which validity?

Answer 33

Content validity

Question 34

Evaluates how accurately a test measures the outcome it was designed to measure, for now or in the future, refers to?

Answer 34

Criterion related validity

Question 35

Which two types of criterion-related vaidity are there?

Answer 35

Concurrent validity (the ability of a test to predict an event in the present) and predictive validity (the ability of a test to measure some event or outcome in the future)

Question 36

Does the test measure the concept that it is intended to measure, refers to which validity?

Answer 36

Construct validity