Ch.3 - Differences, consistency, and the meanings of test scores

Question 1

The Nature of Variability

Question 2

What is the importance of Variability (Differences among people) in Psychology?

Answer 2

It is the most important factor when it comes to Research and Scientific Applications of Psychology
(common example: experiment with two conditions, you want as the experimenter to observe differences among the two conditions. These differences are what lead you to the conclusion that your manipulation has an effect -> lead to discovering something new)

Question 3

What are the two types of variability that behavioral scientists attempt to measure?

Answer 3

• Interindividual variability
• Intraindividual variability

Question 4

Interindividual Variability

Answer 4

The Differences that exist between people
(e.g. differences among students on the SAT score)

Question 5

Intraindividual Variability

Answer 5

The Differences that emerge in one person over time or under different circumstances

Question 6

What are the steps in quantifying psychological differences?

Answer 6

1/. Assume that scores in a psychological test or measure will vary from person to person
2/. Create the distribution of scores: A set of test scores
3/. Quantify the VARIABILITY: Differences among scores in a distribution of scores

Question 7

Variability and Distribution of Scores

Question 8

What is Variance?

Answer 8

A statistical way of quantifying variability or individual differences in a distribution or set of scores

Question 9

What is Covariance?

Answer 9

A statistical way of quantifying the connection between the variability of one set of scores and the variance in another set of scores

Question 10

What are some main concepts present in distribution of scores?

Answer 10

• Central Tendency
• Variability
• Shape

Question 11

What is Central Tendency?

Answer 11

The average/mean of scores (else, the most “typical” score)
(See Picture 1 for Formula)

Question 12

What is Variability?

Answer 12

The Differences among people

Question 13

What is the relationship between Variability, Variance and Standard Deviation?

Answer 13

Variance and SD reflect (in a statistical way) the Variability as the degree to which scores in a distribution deviate from the mean of the distribution
(See Picture 2 for Variance Formula, Picture 3 for SD Formula)

Question 14

What two factors affect the size of the variance?

Answer 14

• Degree to which scores in a distrubution differ from each other (as this degree of difference increases, so does variance)
• The metric of the scores of the distribution (the larger the metric, the larger the variance)
  ~ IQ scores between 80-130, GPA scores between 0.0-4.0. In the same sample, even though participants might have the same degree of difference in IQ and the same degree of difference in GPA scores, the IQ variance will be a lot larger because of differences in how IQ and GPA are measured

Question 15

What are 4 factors to consider when interpreting SD or Variance?

Answer 15

• SD or Variance can never be less than 0.
• There is no simple way to interpret a Variance or SD as large or small (e.g. say variance = 56.63, is this large or small? Depends on metric score, what the typical variability might be for whatever scores are in the distribution)
• The Variance of a distribution of scores is most meaningful when put into a context (e.g. say we have two samples where we measure IQ, it is meaningful to compare the variances of the two samples on their IQ tests, and determine which one has the larger Variance)
• The importance of variance and SD lies mainly in their effects on other values that are more directly interpretable

Question 16

(Distribution) Shapes - General info

Answer 16

• x-axis: score/value on a test/measurement, y-axis: proportion of people who had a specific score on that test.
• Symmetrical distribution: NORMAL distribution (in reality you rarely or never find a normal distribution)
• (See picture 4 for skewed distributions and their names)

Question 17

Quantifying the Association or Consistency between Distributions

Question 18

What are 2 important things to consider about the association between two variables?

Answer 18

• The direction of the association (positive/negative association)
• The magnitude of the association
  (In terms of consistency, a strong/weak association between two variables shows that individual differences are consistent/inconsistent across the two variables)

Question 19

What method of visualizing data is good for visualizing associations and why?

Answer 19

Scatterplots.
They’re good for:
- visually presenting a good sense of that association
- Reveal extreme scores
- Reveal more complicated types of associations apart from the usual linear positive/negative ones

Question 20

What is Covariability?

Answer 20

The degree to which two distributions of scores vary in a corresponding manner

Question 21

What is Covariance?

Answer 21

The Degree of association between the variability in the two distributions of scores
(See Picture 5 for Formula)

Question 22

What are some other general notes on Covariance?

Answer 22

• Serves as a basis for many other statistical concepts and procedures (like variance)
• Only gives us info regarding the direction of the association between two variables, and not the magnitude

Question 23

What is a Variance-Covariance Matrix?

Answer 23

Shows us the variances between two variables.

Question 24

What are the properties of a Variance-Covariance Matrix?

Answer 24

• Each Variable has a row and column
• A Variable’s variance is in the diagonal
• All the other cells present covariances between pairs of variables
• The Matrix is symmetric, all the values below and above the diagonal are identical
  (Picture 6 for an example)

Question 25

What is the correlation coefficient?

Answer 25

A number that provides an easily interpretable number for an association between two variables. (Picture 7 for Formula)
- Gives us info both about direction and magnitude of association
- Also called Pearson correlation

Question 26

Variance and Covariance for Composite Variables

Question 27

What is a composite score?

Answer 27

A composite score is a single score derived from multiple pieces of information. (e.g. if a test has multiple items, specifically 5, a person’s composite score on the test is the sum of the scores that person had on all 5 items)
(See Picture 8 for an example of a composite score table and the formula for Composite score Covariance)

Question 28

How to calculate the variance of a composite score?

Answer 28

the variance of the composite variable = sum of the variances of item + (the correlation between scores on the items * t standard deviations of the two items)
s²_composite = s²_i + s²_j + 2r_ijs_is_j

Question 29

Binary Items

Question 30

What are dichotomous or binary items?

Answer 30

Items that can only take two possible values/categories (e.g. yes or no, male or female, head or tails etc.)

Question 31

How are tests based on binary items typically scored?

Answer 31

Summing or averaging responses across items
(If Depression had 10 items, then we would likely compute a person’s score by summing all 10 responses)

Question 32

Interpreting Test Scores

Question 33

What are two aspects in interpreting the meaning of test scores?

Answer 33

• Basic quantitative meaning of a test score being high or low
• psychological meaning of a test score (what does a high test on a score mean psychologically?)

Question 34

To interpret test scores, we need an interpretive frame to help us make sense of those raw test scores. On what 2 pieces of information is this frame based on?

Answer 34

• Whether the raw score falls below or above the mean
  ~ AND: how much it falls above or below the mean
  (e.g. if you get 40, and the class average is 36, then you know that you got an above average grade)
• Variability within a distribution of raw test scores
  ~ Usually, the SD is used to interpret the meaning of the distance between a raw score and the mean
  (e.g., if you got 40, but there was a big difference among the scores of all test takers, and a mean of 36, your score is likely slightly above average.
  If you got 40, but everybody else got between 34-38, your score is more above average than in the first example, because the class variability is smaller)

Question 35

What are z scores (standard scores)?

Answer 35

They measure the distance between a data point and the mean using standard deviations.
(In other words, they indicate the extremity of a score and towards what direction that score goes)
- Shows you how many SD above or below the mean the raw test score is.
- mean = 0; SD = 1

Question 36

Why are z scores important?

Answer 36

• Provide precise information about the degree to which an individual’s test score is above or below the mean test score
• Important for computing other important statistical values (e.g. correlation coefficient)
  (See picture 9 for Z-score formula)

Question 37

What are the benefits of Z scores?

Answer 37

• They express test scores in a way that bypasses the ambiguity of many measures (by framing the meaning of a score in terms of how far away it is from a mean)
• Can be used to compare scores across tests that are on different sized units

Question 38

Some other important notes on the z scores

Answer 38

• They express a score in terms of its relation to an entire distribution of scores, not in an absolute sense (e.g., in the case of a grade on a test, your z score of +2 tells us something about your score in relation to the rest of the class)
• Is useful to conceptualize and compute some important statistical values
  (e.g. It is often difficult to examine the
  consistency of raw scores when those scores are values expressed in different metric units -> By transforming each set of values to z scores, we express both sets of scores with a common metric)

Question 39

What are some reasons test users and takers might struggle to interpret z scores?

Answer 39

• If respondents’ test scores would be expressed as negative numbers (how do you interpret a negative value on self-esteem or neuroticism?)
• Scores are sometimes expressed in decimals or fractions
  In these cases, scientists use converted standard scores (or else, standardized scores)

Question 40

What are Converted standard scores?

Answer 40

Z scores that have been converted to values that people might find easier to understand
(Again, express how much a raw test score is above or below the mean)
Sometimes called standardised scores

Question 41

How do you obtain a converted standard score?

Answer 41

• Select a new mean and standard deviation for the distribution of converted scores
• Apply formula in Picture 10

Question 42

What are percentile ranks?

Answer 42

Another way of interpreting and presenting test results. They indicate the percentage of scores that are below a specific test score

Question 43

How do you calculate a percentile rank?

Answer 43

• 1st way: e.g. 75 people taking a test, Person 1 obtains a score of 194. We see how many people scored below 194. E.G. those people are 52 -> 52/75 = 0.69 (69%), so Person 1 scored at the 69th percentile.
• 2nd way: If we only have info regarding the mean and SD of raw test scores, we can compute a z score for that raw test score and link it to a percentile. IF WE CAN ASSUME THAT INDIVIDUAL DIFFERENCES ARE NORMALLY DISTRIBUTED, we can link standard scores to the standard normal distribution (mean of 0, sd of 1)

Question 44

How do we use the standard normal distribution?

Answer 44

• Websites/Apps
• Tables (as in Stats book): We use z score and the table gives us immediately the percentile rank of that z score

Question 45

What are normalized scores?

Answer 45

They are adjusted values, initially measured on different scales and transcribed to one common scale.
!!! Assumes that scores taken from test developers are distributed normally !!!

Question 46

If the scores obtained from test makers are not normally distributed, how do scientists solve for this problem?

Answer 46

They try to transform their non-normal distribution into one that approximates a normal distribution.
- First they assume the following:
~ levels of the psychological attribute are indeed normally distributed,
~ the actual test data that they obtained in their sample are imperfect reflections of the distribution of the construct.
!!! This process is called normalization process !!!

Question 47

What are the 3 steps in the normalization process?

Answer 47

1) compute direct percentile ranks from the raw test scores
2) convert percentile ranks into z scores
3) take z score and convert it into a converted standard score onto the metric we would like

Question 48

Test Norms

Question 49

Why do scientists often norm their tests?

Answer 49

In order to facilitate their interpretation by users

Question 50

What is the process of normalization?

Answer 50

Test developers administer their new test to a large group of people who are believed to represent a larger population of people. After this large group has taken the test and their scores have been calculated, test users can use their scores as a frame of reference for interpreting the scores of other people who will eventually take the test. The large group of people used in the construction of a test is referred to as the reference sample, and their scores are called the “norms” for the test.

Question 51

What is the value of the normed data dependent on?

Answer 51

The extent to which the reference sample is representative of the population
- Selecting a sample which is representative of the population can be done through probability sampling: use of certain procedures that ensure a representative sample
(nonprobability sampling: procedures that are not likely to produce a representative sample)