WEEK 1: INTERPRETING STATISTICAL RESULTS

Question 1

Define: p-value

Answer 1

measures the probability of obtaining the observed results, assuming that the null hypothesis is true

Tells us there’s a difference between groups and how likely it is due to chance

example: if the p value is less than 0.05 it means there’s a less than 5% chance the result occured due to chance

A low or high p-value does not prove anything with regard to the effectiveness of an intervention

Question 2

Define effect size

Explain its used

Describe how it can be displayed

Answer 2

Effect size: statistic which estimates the magnitude of an effect (e.g. mean difference, regression coefficient, Cohen’s d, correlation coefficient)

Can be used as a relevant interpretation of an estimated magnitude (weak, moderate, or strong-effect)

An effect size can be displayed both as an unstandard and standardized value

Question 3

Describe unstandardized and standardized effect sizes

Answer 3

If the original units of measurements are meaningful, the presentation of unstandardized effect statistics is preferable over that of standardized effect statistics

Examples: metres, degrees, velocity

Standardized effect statistics are always calculable if sample size and standard deviation are given along with unstandardized effect statistics

Examples: percentage, Cohen’s d, Hedge’s g

Question 4

Differentiate between minimal detectable change (MDC) and minimal clinically important difference (MCID)

Answer 4

Minimal Detectable Change (MDC): The minimum amount of change in a patient’s score that ensures the change isn’t the result of measurement error.
- Statistical significance

The smallest difference in score in the domain of interest which patients perceive as beneficial and which would mandate, in the absence of troublesome side effects and excessive cost, a change in the patient’s management

Question 5

Define: Cohen’s d

Answer 5

indicate the standardized difference between two means.

can be used to quantify effect size

Small effect: d = 0.2 – 0.49
Medium effect: d = 0.5 – 0.79
Large effect: d ≥ 0.8

Question 6

Define: Pearson Correlation Coefficient

Answer 6

A correlation analysis provides a quantitative way of measuring the strength of a relationship between two variables

Negative relationship: -1
No relationship: 0
Positive relationship: 1

Interpreting effect size
* Small = +- 0.1
* Medium = +- 0.3
* Large = +- 0.5

Question 7

Define: intraclass correlation coefficient

Answer 7

• Used to measure rater reliability for continuous data

Interpreting ICC scores
Poor < 0.5
Moderate = 0.5 to 0.75
Good = 0.75 to 0.9
Excellent > 0.9

Question 8

Define: variance

Answer 8

Provides an estimate of the degree of scatter of individual sample data points about the sample mean - How close an individual value is to the mean value

Question 9

Why are effect deviations squared?

Answer 9

Because individual data points fall above and below the mean, the effect of direction of difference will cause some deviations from the mean to be positive and some to be negative

To overcome this, effect deviations are squared to obtain a positive number

Question 10

State the population variance and sample variance equations and describe each variable

Answer 10

• 𝜇 = population mean
• 𝑋( = sample mean
• Xi = each individual data point
• N = total number of data points in a population
• n = total number of data points in a sample

Question 11

Define: range

Answer 11

Difference between the
maximum and minimum values

Question 12

Define: standard deviation

Answer 12

square root of variance

Question 13

Define: interquartile range

Answer 13

Range of
the middle 50% of the data.

Question 14

Define: coefficient of variation

State eqquation

Explain usage

Answer 14

Coefficient of variation: the ratio of the standard deviation to the mean; shows the extent or variability in relation to the mean

CV = (SD/mean) x100

Question 15

Define: standard error of the mean (SEM)

State the equation

Answer 15

Standard Error of the Mean: estimate the precision or reliability of a sample, as it relates to the population from which the sample is drawn

SEM does not provide an estimate of the scatter of sample data about the sample mean, and should not be used as such

The accuracy of each sample is determined by the number of observations, therefore error decreases as sample size increases

SEM = s/ sqrt N

where s = standard deviation

N = sample size

Question 16

Define: confidence intervals

State equation

Answer 16

range of values that encompass the actual population or ‘true’ value, with a given probability. The width of the interval indicates the precision of the estimate (effect size), the wider the interval, the less precise the estimated effected size. A study with a small sample size will have greater random error, leading to a wider interval

CI = mean +- [(alpha/tails x critical value) x SEM]

Question 17

Define: alpha levels

Answer 17

• probability, the researcher is willing to accept, that the findings are a result of sampling error
• It is determined by the level of confidence the researcher decides to use, and is typically set to either 90%, 95% or 99%
• The greater the confidence level, the wider the interval
• Sample size can also affect the width of the interval, with smaller
  sample sizes leading to wider intervals
• A confidence of 95% will provide an alpha level of 0.05

Question 18

Define: tails

Answer 18

• The number of tails depends on the question the researcher is asking
• Two tails are used if the researcher would like to know if the results of
  an intervention differ from a control or alternate group
• It is common to use 2 tails in intervention-based research
• If we are certain the results will only go in one direction, you can use one tail (example: you’re certain the results from the test will only be positive)

Question 19

When do you use a z distribution and t distribution

Answer 19

• The z-distribution is used when the variance of the population is known
  In some cases, an author may choose to use the z-distribution if the sample size is greater than 30
• The t-distribution should be used when the true variance is not known and has been estimated from the sample. With larger sample sizes, a t-distribution value will become similar to that of a z-distribution

Question 20

Define: content validity

Answer 20

assesses whether a test is representative of all aspects of the construct (is the test giving you the information important to you?)

Question 21

Give an example of poor content validity

Answer 21

A measure designed to assess lower extremity functional status that inquires about walking 2-blocks and nothing more does not possess content validity

Many other important ambulatory activities (negotiating stair, running, rising from sitting etc.) were not included.

Question 22

Define: construct validity

Answer 22

Refers to whether you can draw inferences about test scores related to the concept being studied.

For example, if a person has a high score on a survey that measures anxiety, does this person truly have a high degree of anxiety?

Graph example from class. A new physiotherapy assessment tool has very close score values to a gold standard assessment

Question 23

How can we demonstrate construct validity?

Answer 23

Homogeneity: The instrument measures one construct. The construct must be well defined

Convergence: this occurs when the instrument measures concepts similar to that of other instruments.

Theory evidence: This is evident when behaviour is similar to theoretical propositions of the construct measured in the instrument
- For example, when an instrument measures anxiety, one would expect to see that participants who score high on the instrument for anxiety also demonstrate symptoms of anxiety in their day- to-day lives

Question 24

Define: criterion validity

Answer 24

evaluates how accurately a test measures the outcome it was designed to measure.

Correlations can be conducted to determine the extent to which the different instruments measure the same variable.

Question 25

Define some measurements of criterion validity

Answer 25

convergent validity: shows that an instrument is highly correlated with instruments measuring similar variables

divergent validity: shows that an instrument is poorly correlated to instruments that measure different variables.

predictive validity: means that the instrument should have high correlations with future criterions. For example, a score of high self-efficacy related to performing a task should predict the likelihood a participant completing the task.

Question 26

Define: inter-rater reliability

Answer 26

reflects the variation between 2 or more raters who measure the same group of subjects.

Question 27

Define: intra-rater reliability

Answer 27

It reflects the variation of data measured by 1 rater across 2 or more trials.

Question 28

Define: test-retest reliability

Answer 28

It reflects the variation in measurements taken by an instrument on the same subject under the same conditions. It is generally indicative of reliability in situations when raters are not involved or rater effect is neglectable, such as self-report survey instrument

Question 29

Define: absolute reliability

Answer 29

quantifies random measurement error in the same units as the original measurement

• Thus, measurement error for range of motion is reported in degrees; measurement error for the 6-minute walk test is stated in meters; and measurement error for the LEFS is expressed in LEFS points

Absolute reliability provides an estimate of the variability among multiple measurements for a truly stable or unchanged patient
* As such it comments on the consistency aspect of reliability.

Question 30

Define: relative reliability

Answer 30

Relative reliability examines the extent to which a measure is capable of differentiating among the objects of measurement

Example: differentiating between a grade 1 and grade 2 ligament sprain

Question 31

Define: sensitivity

Answer 31

• The ability of a test to reliably detect the presence of disease (positivity in disease).
• Less likely to miss the condition if one is present
• Sensitivity = True Positive / (True Positives + False Negatives)
• High sensitivity = low false negatives

Example: the ability of an x ray to detect stress fractures - low sensitivity

Higher on a bone scan than on an xray

Question 32

Define: specificity

Answer 32

• The ability of a test to reliably detect the absence of a specific disease (negativity in health)
• Less likely to say the patient has the condition when the really do not
• High specificity means you won’t waste resources
• Low specificity, likely to send for imaging when not needed
• Specificity = True Negatives / (True Negatives + False Positives)
• High specificity = low false positives

Higher on an x ray than a bone scan

Question 33

Can you use sensitivity and specificity to predict whether an indvidual is diseased or disease free?

Answer 33

No - Sensitivity and specificity are merely properties of a test. Sensitivity and specificity should not be used to make predictive statements about an individual patient.

Question 34

Define: positive predictive value (PPV)

Answer 34

• A positive predictive value (PPV) is useful to indicate the proportion of individuals who actually have the disease when the diagnostic test indicates the presence of that disease
• PPV = True Positive / (True Positive + False Positive)
• PPV = True Positive / All Positive Results

Question 35

Define: negative predictive value (NPV)