Reliability and Validity

Question 1

What is meant by reliability?

Answer 1

The extent to which results or a measure are consistent

Question 2

What is meant by internal reliability?

Answer 2

Whether the first half of a tests results are consistent with the the second half’s.

Question 3

What is meant by external reliability?

Answer 3

The ability to reproduce the same results each time the test is carried out.

Question 4

How to improve reliability in an experiment?

Answer 4

1)Standardisation-Making all the procedure exactly the same for each time the test is carried out through a script etc
2)Take multiple measures and find an average
3)Pilot studies beforehand check the ambiguity of questions and allows for rewording of such questions and timings etc

Question 5

How do you assess reliability of an experiment?

Answer 5

Test-retest involving the ppts repeating the same questionnaire or assessment multiple times with gap in between (e.g a week) and finding whether correlation of the results is strongly positive

Question 6

How do you improve reliability of an observation?

Answer 6

1) Behavioural categories must be operationalised and used consistently across the study
2)Pilot studies allow for timings to be checked, behavioural categories to be made less ambiguous and other aspects of the procedure to be checked
3) Standardised procedure so that every repetition is identical

Question 7

How do you assess reliability of an observation?

Answer 7

Inter-rater reliability where two researchers observe the same activity and use the same pre-agreed behavioural categories and compare the results. a correlation coefficient of +0.8 indicates reliable results

Question 8

How do you improve the reliability of self report methods?

Answer 8

1)reduce ambiguity of questions
2)pilot studies to make sure questions are less ambiguous
3)standardise the questions

Question 9

How do you assess the reliability of self report methods?

Answer 9

1)test-retest repeating tests with time inbetween
2) splithalf
3) inter-interviewer reliability

Question 10

What is meant by validity?

Answer 10

Validity is the extent to which a measure accurately measures what it claims to and how much it can be generalised outside of the lab setting

Question 11

What is internal validity?

Answer 11

This is the extent to which a test measures what is claims to measure

Question 12

What is meant by demand characteristics and how do they impact internal validity?

Answer 12

Demand characteristics indicate the aims and hypothesis of a study to the ppts causing them to change there behaviour meaning the test no longer tests what is claims to

Question 13

What meant by experimenter bias and how does it impact internal validity?

Answer 13

Experimenter bias refers to the preconceptions and interpretations effect the results displaying the researchers bias

Question 14

What are ppt variables and how do they impact internal validity?

Answer 14

These refer to characteristics of the ppts that impact the dependent variable and mean the test no longer measures what it claims to

Question 15

What is external validity?

Answer 15

This refers to the extent to which results can be generalised outside of the lab setting

Question 16

What is temporal validity?

Answer 16

This refers to how the period that the study effects the internal validity as certain cultural norms have changed

Question 17

what is population validity?

Answer 17

This is how representative of the target population a sample is and how it affects the generalisability of the results

Question 18

What is ecological validity?

Answer 18

This is the extent to a which findings can be generalised to situations outside of the research setting

Question 19

How do we assess internal validity?

Answer 19

1) Face validity - a common sense judgment
2) Concurrent validity - two tests of the same characteristic display the same/similar results
3) Content validity - other experts assess the degree to which a test measures what it claims to

Question 20

How do we improve the internal validity of a questionnaire?

Answer 20

By revising and consequently removing irrelevant questions

Question 21

How do we improve the internal validity of an experiment?

Answer 21

Control for:
Investigator effects - standardise etc
Demand characteristics
Confounding variables
Social desirability

Question 22

How do we improve the validity of an experiment?

Answer 22

1)Single blind method - reduces demand characteristics as ppts do not know what conditions mean
2)double blind method - reduces both demand characteristics and investigator bias by having neither ppts or researcher know what each condition means

Question 23

Other methods of improving validity?

Answer 23

1) Standardisation of procedure and questions
2) Operationalised variables
3) Control of extraneous variables such as ppt variables