Ch.3 - Differences, consistency, and the meanings of test scores Flashcards
The Nature of Variability
What is the importance of Variability (Differences among people) in Psychology?
It is the most important factor when it comes to Research and Scientific Applications of Psychology
(common example: experiment with two conditions, you want as the experimenter to observe differences among the two conditions. These differences are what lead you to the conclusion that your manipulation has an effect -> lead to discovering something new)
What are the two types of variability that behavioral scientists attempt to measure?
- Interindividual variability
- Intraindividual variability
Interindividual Variability
The Differences that exist between people
(e.g. differences among students on the SAT score)
Intraindividual Variability
The Differences that emerge in one person over time or under different circumstances
What are the steps in quantifying psychological differences?
1/. Assume that scores in a psychological test or measure will vary from person to person
2/. Create the distribution of scores: A set of test scores
3/. Quantify the VARIABILITY: Differences among scores in a distribution of scores
Variability and Distribution of Scores
What is Variance?
A statistical way of quantifying variability or individual differences in a distribution or set of scores
What is Covariance?
A statistical way of quantifying the connection between the variability of one set of scores and the variance in another set of scores
What are some main concepts present in distribution of scores?
- Central Tendency
- Variability
- Shape
What is Central Tendency?
The average/mean of scores (else, the most “typical” score)
(See Picture 1 for Formula)
What is Variability?
The Differences among people
What is the relationship between Variability, Variance and Standard Deviation?
Variance and SD reflect (in a statistical way) the Variability as the degree to which scores in a distribution deviate from the mean of the distribution
(See Picture 2 for Variance Formula, Picture 3 for SD Formula)
What two factors affect the size of the variance?
- Degree to which scores in a distrubution differ from each other (as this degree of difference increases, so does variance)
- The metric of the scores of the distribution (the larger the metric, the larger the variance)
~ IQ scores between 80-130, GPA scores between 0.0-4.0. In the same sample, even though participants might have the same degree of difference in IQ and the same degree of difference in GPA scores, the IQ variance will be a lot larger because of differences in how IQ and GPA are measured
What are 4 factors to consider when interpreting SD or Variance?
- SD or Variance can never be less than 0.
- There is no simple way to interpret a Variance or SD as large or small (e.g. say variance = 56.63, is this large or small? Depends on metric score, what the typical variability might be for whatever scores are in the distribution)
- The Variance of a distribution of scores is most meaningful when put into a context (e.g. say we have two samples where we measure IQ, it is meaningful to compare the variances of the two samples on their IQ tests, and determine which one has the larger Variance)
- The importance of variance and SD lies mainly in their effects on other values that are more directly interpretable
(Distribution) Shapes - General info
- x-axis: score/value on a test/measurement, y-axis: proportion of people who had a specific score on that test.
- Symmetrical distribution: NORMAL distribution (in reality you rarely or never find a normal distribution)
- (See picture 4 for skewed distributions and their names)
Quantifying the Association or Consistency between Distributions
What are 2 important things to consider about the association between two variables?
- The direction of the association (positive/negative association)
- The magnitude of the association
(In terms of consistency, a strong/weak association between two variables shows that individual differences are consistent/inconsistent across the two variables)
What method of visualizing data is good for visualizing associations and why?
Scatterplots.
They’re good for:
- visually presenting a good sense of that association
- Reveal extreme scores
- Reveal more complicated types of associations apart from the usual linear positive/negative ones
What is Covariability?
The degree to which two distributions of scores vary in a corresponding manner
What is Covariance?
The Degree of association between the variability in the two distributions of scores
(See Picture 5 for Formula)
What are some other general notes on Covariance?
- Serves as a basis for many other statistical concepts and procedures (like variance)
- Only gives us info regarding the direction of the association between two variables, and not the magnitude
What is a Variance-Covariance Matrix?
Shows us the variances between two variables.
What are the properties of a Variance-Covariance Matrix?
- Each Variable has a row and column
- A Variable’s variance is in the diagonal
- All the other cells present covariances between pairs of variables
- The Matrix is symmetric, all the values below and above the diagonal are identical
(Picture 6 for an example)
What is the correlation coefficient?
A number that provides an easily interpretable number for an association between two variables. (Picture 7 for Formula)
- Gives us info both about direction and magnitude of association
- Also called Pearson correlation
Variance and Covariance for Composite Variables
What is a composite score?
A composite score is a single score derived from multiple pieces of information. (e.g. if a test has multiple items, specifically 5, a person’s composite score on the test is the sum of the scores that person had on all 5 items)
(See Picture 8 for an example of a composite score table and the formula for Composite score Covariance)
How to calculate the variance of a composite score?
the variance of the composite variable = sum of the variances of item + (the correlation between scores on the items * t standard deviations of the two items)
s2composite = s2i + s2j + 2rijsisj
Binary Items
What are dichotomous or binary items?
Items that can only take two possible values/categories (e.g. yes or no, male or female, head or tails etc.)
How are tests based on binary items typically scored?
Summing or averaging responses across items
(If Depression had 10 items, then we would likely compute a person’s score by summing all 10 responses)
Interpreting Test Scores
What are two aspects in interpreting the meaning of test scores?
- Basic quantitative meaning of a test score being high or low
- psychological meaning of a test score (what does a high test on a score mean psychologically?)
To interpret test scores, we need an interpretive frame to help us make sense of those raw test scores. On what 2 pieces of information is this frame based on?
- Whether the raw score falls below or above the mean
~ AND: how much it falls above or below the mean
(e.g. if you get 40, and the class average is 36, then you know that you got an above average grade) - Variability within a distribution of raw test scores
~ Usually, the SD is used to interpret the meaning of the distance between a raw score and the mean
(e.g., if you got 40, but there was a big difference among the scores of all test takers, and a mean of 36, your score is likely slightly above average.
If you got 40, but everybody else got between 34-38, your score is more above average than in the first example, because the class variability is smaller)
What are z scores (standard scores)?
They measure the distance between a data point and the mean using standard deviations.
(In other words, they indicate the extremity of a score and towards what direction that score goes)
- Shows you how many SD above or below the mean the raw test score is.
- mean = 0; SD = 1
Why are z scores important?
- Provide precise information about the degree to which an individual’s test score is above or below the mean test score
- Important for computing other important statistical values (e.g. correlation coefficient)
(See picture 9 for Z-score formula)
What are the benefits of Z scores?
- They express test scores in a way that bypasses the ambiguity of many measures (by framing the meaning of a score in terms of how far away it is from a mean)
- Can be used to compare scores across tests that are on different sized units
Some other important notes on the z scores
- They express a score in terms of its relation to an entire distribution of scores, not in an absolute sense (e.g., in the case of a grade on a test, your z score of +2 tells us something about your score in relation to the rest of the class)
- Is useful to conceptualize and compute some important statistical values
(e.g. It is often difficult to examine the
consistency of raw scores when those scores are values expressed in different metric units -> By transforming each set of values to z scores, we express both sets of scores with a common metric)
What are some reasons test users and takers might struggle to interpret z scores?
- If respondents’ test scores would be expressed as negative numbers (how do you interpret a negative value on self-esteem or neuroticism?)
- Scores are sometimes expressed in decimals or fractions
In these cases, scientists use converted standard scores (or else, standardized scores)
What are Converted standard scores?
Z scores that have been converted to values that people might find easier to understand
(Again, express how much a raw test score is above or below the mean)
Sometimes called standardised scores
How do you obtain a converted standard score?
- Select a new mean and standard deviation for the distribution of converted scores
- Apply formula in Picture 10
What are percentile ranks?
Another way of interpreting and presenting test results. They indicate the percentage of scores that are below a specific test score
How do you calculate a percentile rank?
- 1st way: e.g. 75 people taking a test, Person 1 obtains a score of 194. We see how many people scored below 194. E.G. those people are 52 -> 52/75 = 0.69 (69%), so Person 1 scored at the 69th percentile.
- 2nd way: If we only have info regarding the mean and SD of raw test scores, we can compute a z score for that raw test score and link it to a percentile. IF WE CAN ASSUME THAT INDIVIDUAL DIFFERENCES ARE NORMALLY DISTRIBUTED, we can link standard scores to the standard normal distribution (mean of 0, sd of 1)
How do we use the standard normal distribution?
- Websites/Apps
- Tables (as in Stats book): We use z score and the table gives us immediately the percentile rank of that z score
What are normalized scores?
They are adjusted values, initially measured on different scales and transcribed to one common scale.
!!! Assumes that scores taken from test developers are distributed normally !!!
If the scores obtained from test makers are not normally distributed, how do scientists solve for this problem?
They try to transform their non-normal distribution into one that approximates a normal distribution.
- First they assume the following:
~ levels of the psychological attribute are indeed normally distributed,
~ the actual test data that they obtained in their sample are imperfect reflections of the distribution of the construct.
!!! This process is called normalization process !!!
What are the 3 steps in the normalization process?
1) compute direct percentile ranks from the raw test scores
2) convert percentile ranks into z scores
3) take z score and convert it into a converted standard score onto the metric we would like
Test Norms
Why do scientists often norm their tests?
In order to facilitate their interpretation by users
What is the process of normalization?
Test developers administer their new test to a large group of people who are believed to represent a larger population of people. After this large group has taken the test and their scores have been calculated, test users can use their scores as a frame of reference for interpreting the scores of other people who will eventually take the test. The large group of people used in the construction of a test is referred to as the reference sample, and their scores are called the “norms” for the test.
What is the value of the normed data dependent on?
The extent to which the reference sample is representative of the population
- Selecting a sample which is representative of the population can be done through probability sampling: use of certain procedures that ensure a representative sample
(nonprobability sampling: procedures that are not likely to produce a representative sample)
