L3- Reliability- Assessing and improving reliability

Question 1

State the ways in which you can assess the reliability in an observation

Answer 1

1) Test-retest
2) Inter observer (rater) reliability
3) Pilot study

Question 2

Describe each of the ways in which reliability can be tested in an observation

Answer 2

Test retest:

• Repeat observation 2nd time using same pps
• Compare results gained from 2nd observation with results gained from 1st observation (easier if observation recorded)
• Results from 1st and 2nd observation should be very similar and should produce a ➕ correlation coefficient of +0.8 or more in order to be reliable

Inter observer (rater) reliability:

• To make sure observation ✖️ biased- use ⬆️ than 1 observer/psychologist to observe & record 🏃‍♂‍ separately
• Results from both psychologists compared and should gain a ➕ correlation coefficient of +0.8 or more if observation reliable

Pilot study:

• Conduct small trial run of observation before main research study carried out
• ensures that procedures and resources used in research improve precision when measuring 🏃‍♂‍ - ensures procedures planned properly … minimises human error and variation
• 🔑 concepts and terms need to be operationalised so that all researchers know what they are defining, measuring and observing- … makes sure all observers been properly trained to look for appropriate 🏃‍♂‍ that fits into each category

Question 3

State the ways in which you can improve the reliability in an observation

Answer 3

1) Operationalise behavioural 🏃‍♂‍ categories properly

2) Training

Question 4

Describe each of the ways in which reliability can be improved in an observation

Answer 4

Operationalise behavioural 🏃‍♂‍ categories properly:

• If Inter observer reliability ⬇️- then reliability could improved by ensuring 🏃‍♂‍ categories operationalised properly and clearly
• Ensures each observer understands categories properly when recording data -ALSO observers might need further training about which 🏃‍♂‍ to observe and how they should be measured

Training:

• If results from pilot study ✖️ very clear- then reliability could improved by giving ⬆️ training and practice to observers
• allows them to become ⬆️ familiar with the 🏃‍♂‍ categories and … can respond ⬆️ quickly when observing pps

Question 5

State the ways in which you can assess the reliability in a self-report interview/questionnaire

Answer 5

1) Test retest

2) Inter researcher reliability

Question 6

Describe each of the ways in which reliability can be assessed in a self-report interview/questionnaire

Answer 6

Test retest:

• Give self-report to group of pps and collect results
• Give same pps same self-report to complete a 2nd time (short interval of ⏰ should be left between 1st and 2nd test)
• Compare results from 1st self-report to 2nd self-report and should be very similar/consistent
• Should gain ➕ correlation coefficient of +0.8 or more if the self-report reliable

Inter researcher reliability:

• To make sure self-report ✖️ biased use ⬆️ than 1 psychologist to interview pps and record answers separately
• Researchers need to act in similar and consistent ways- carry out procedure and design in exactly the same way to make research consistent
• Results from both psychologists should be compared and should gain a ➕ correlation coefficient of +0.8 or more if interview reliable

Question 7

State the ways in which you can improve the reliability in a self-report interview/questionnaire

Answer 7

1) Questions used in the interview

Question 8

Describe each of the ways in which reliability can be improved in a self-report interview/questionnaire

Answer 8

Questions used in the interview:

• must make sure interview questions ✖️ ambiguous
• clear so that pps understand them and can give same answers if questions asked again in future
• Ambiguous questions rewritten if necessary or removed

Question 9

State the ways in which you can assess the reliability in an experiment

Answer 9

1) Test-retest

Question 10

Describe each of the ways in which reliability can be assessed in an experiment

Answer 10

Test retest:

• Conduct experiment once and collect results
• Repeat experiment again a few weeks later with same pps- tested in exactly the same way
• Compare results gained from both occasions
• Results should be similar- correlation coefficient of +0.8 or more should be gained

Question 11

State the ways in which you can improve the reliability in an experiment

Answer 11

1) Standardisation of instructions

Question 12

Describe each of the ways in which reliability can be improved in an experiment

Answer 12

Standardisation of instructions:

• If experiment conducted twice- then procedures are repeated twice
• Procedures should be exactly the same for each pps that takes part in experiment (standardised instructions)
• helps ensure reliable results gained
• 🔑 concepts and variables should also be operationalised