chapter 7 - reliability

Question 1

true score

Answer 1

he true score is a theoretical value that represents a test taker’s score without error. If a person took parallel forms of a test thousands of times (assuming no practice or fatigue effects), the mean of all their scores would be a good approximation of their true score since the error would be almost entirely averaged out. It should be distinguished from validity.

Essentially, true score theory maintains that every measurement is an additive composite of two components: true ability (or the true level) of the respondent on that measure; and random error.

Question 2

forms of reliability

Answer 2

Reliability refers to the consistency of a measure. Psychologists consider three types of consistency: over time (test-retest reliability), across items (internal consistency), and across different researchers (inter-rater reliability)

1. Test-retest reliability
   The test-retest reliability method in research involves giving a group of people the same test more than once. If the results of the test are similar each time you give it to the sample group, that shows your research method is likely reliable and not influenced by external factors, like the sample group’s mood or the day of the week. Here are the guidelines for this type of research:
2. Parallel forms reliability
   This strategy involves giving the same group of people multiple types of tests to determine if the results stay the same when using different research methods. If they do, this means the methods are likely reliable because, otherwise, the participants in the sample group may behave differently and change the results. For this strategy to succeed, it’s important that:
3. Inter-rater reliability
   The inter-rater reliability testing involves multiple researchers assessing a sample group and comparing their results. This can help them avoid influencing factors related to the assessor, including:
   Personal bias
   Mood
   Human error
4. Internal consistency reliability
   Checking for internal consistency in research involves making sure your internal research methods or parts of research methods deliver the same results. There are two typical ways to make this determination:
   Split-half reliability test: You can perform this test by splitting a research method, like a survey or test, in half, delivering both halves separately to a sample group, then comparing the results to ensure the method can produce consistent results. If the results are consistent, then the results of the research method are likely reliable.
   Inter-item reliability test: With this assessment, you administer sample groups multiple testing items, like with parallel forms reliability testing, and calculate the correlation between the results of each of the method results. With this information, you calculate the average and use the number to determine if the results are reliable.

Question 3

measurement error

Answer 3

Measurement Error (also called Observational Error) is the difference between a measured quantity and its true value. It includes random error (naturally occurring errors that are to be expected with any experiment) and systematic error (caused by a mis-calibrated instrument that affects all measurements).

EX: Absolute Error: the amount of error in your measurement. For example, if you step on a scale and it says 150 pounds but you know your true weight is 145 pounds, then the scale has an absolute error of 150 lbs – 145 lbs = 5 lbs.

Question 4

factors of reliability

Answer 4

Reliability is affected by many factors, but from the researcher’s point of view, the three most important factors are the length (or total number of questions), the quality of the questions, and the fit to the group being measured.