17 - Reliability And Validity

Question 1

Reliability

Answer 1

Refers to the consistency, accuracy and stability of findings that enable them to be replicated.
=> degree to which there is no measurement error

R = 1 - True Variance/ Observed variance

Question 2

Error

Answer 2

• random fluctuation error: the extent to which a score has been influenced by irrelevant factors or chance
• systematic error: the effect of unwanted variables biasing values in one direction only

Question 3

Error vs reliability

Answer 3

Increase error decrease reliability

Question 4

Perfect reliability

Answer 4

When the true variance equal the observed variance and error equal 0 and r=1

Question 5

Nil reliability

Answer 5

R=0, when the observed variance is entirely error.

Question 6

Tests to determine reliability

Answer 6

• test-retest reliability measure
• parallel forms reliability measure
• split half reliability
• internal consistency method

Question 7

Test retest reliability method

Answer 7

Obtains a coefficient of temporal stability from observations that were made twice with an intervening time period.

Assume constant experiment environment during both observations.

Risk if the period is too long: maturational factors.

Question 8

Parallel forms method

Answer 8

Requires two equivalent forms of assessment that are administered to respondents on the same occasion.

Pros:

• more accurate estimate or reliability because it is based on larger samplings.
• very minimal time laps and difference

Cons:

• time
• money
• difficulty to build // forms

Question 9

Split half reliability

Answer 9

Splitting the assessment into equal halves.

Reduce the potential for random and systematic error

Question 10

Spearman-brown formula

Answer 10

Correlation equal

r= rN/[1+(N-1)r]

N is the ratio of the length of the desired assessment to the length of the present assessment.

Question 11

Internal consistency method

Answer 11

Computes a correlation coefficient for every conceivable way of dividing the test into 2 and average them out.
- KR-20 for dichotomous data
- conbach’s alpha for scale data if > 0,7 homogeneity
=> fail to measure temporal instability

Question 12

Factors that influence reliability

Answer 12

• length of assessment
• method for estimating it
• type of measurement (observations vs objective measures)
• standardisation and administration (standardised)

Question 13

Standard error of the measurement for error coefficient

Answer 13

SE= SD*sqrd(1-r)

Question 14

Validity

Answer 14

Related to the appropriateness of the measure.

• external validity: extent to which results are transferable to a pop
• internal validity: extent to which the conditions within the experiment are controlled.

Ps: if it is unreliable it cannot be valid

Question 15

External validity

Answer 15

• population validity: sample of participants = target sample
• ecological validity: experimental studies undertaken under lab conditions do not reflect reality

Question 16

Internal validity

Answer 16

• content validity: content measured = intended content to be investigated?
• face validity: public relation validity (disguise real intents or no?)
• predictive validity: difference between obtained and real
• concurrent validity: current scores to present performance
• construct validity

Question 17

Threats to external validity

Answer 17

(Generalisation)

• selection bias
• lack of representativeness
• reactive bias
• fail to describe and operationalise DV and IVs

Question 18

Threats to internal validity

Answer 18

(Control)

• small samples
• violated assumptions
• subjects heterogeneity
• historical actions influencing results
• maturation
• dropout