Chapter 2 Flashcards
what was Anne Anatasi’s definition of a psychological test?
an objective and standardized measure of a sample of behaviour
what is a sample of behaviour?
refers to an individuals performance on a task agreed upon beforehand
what were the four main elements of Anne Anatasi’s definition of a psychological test?
- tests are about behaviour of person that can be observable/non-obserable
- test measures a sample of behaviour
- tests should be objective and standardized
- standardized tests produce standardized scores (ex. z scores)
what is generalizability theory?
extent to which test results vary
what is a sample?
a set of observations/scores/measurements collected
are sample of behaviour tests always accurate?
no, not always representative
you could have a bad day, or have extraneous factors
how can tests be objective and standardized?
- when questions are equal (either same, taken from bank, or all of equivalent difficulty)
- when tests are scored in a consistent manner
depends on: purpose of test, what it is trying to measure, method of measurement
what is a z-score
a number ranging from -3 to +3 which expresses how well you did relative to others
tells who the proportion of entire sample that falls between 2 z scores
what would a z score of +3, +1, and 0.5 tell you?
+3 = much better than everyone else in group (99% better)
+1 = higher than most in group (72% better)
0.5 = central part of distribution of scores (where most people score/average)
when is a test objective
- when it is not influenced by beliefs of person making test
- when Q’s do not unfairly disadvantage test taker
- when test is free of bias
- depends on what test should measure and how well it does at that
what was Lee Cronbach’s definition of a psychological test?
a systematic procedure for comparing the behaviour of 2 people
relinquishes idea of scoring tests in a standardized way
emphasizes that purpose of test is to compare individuals
how was the field of psychometrics founded?
1935 - psychometric society founded by Louis Leon Thurstone who wanted to emphasize the mathematical foundations of psychological research
define psychometrics
psych field concerned with theory and methods of psychological measurement
what are the two main elements of psychometrics?
construction of instruments + procedures for measurement (involves measuring traits, characteristics, symptoms, etc.)
development/refinement of theoretical approaches to measurement (used to evaluate statistical properties of questionnaires, or accuracy of scorers judgements)
what is a construct and who came up with it?
idea/concept that a researcher would like to measure
Lee Cronbach and Paul Meehl (1995)
what are some of the elements of constructs?
include attributes that character a person (ex. shyness, intelligence, etc.) which are reflected in test scores
are generally unobservable
measuring a construct involves creating a set of measures (operationalized measure)
what are nomological networks and who came up with them?
Lee Cronbach and Paul Meehl
specifies what qualities/attributes a construct refers to and how that construct is related o other constructs
also what operations would be used to measure the qualities that define a construct
what are the two types of definitions that every construct has?
formal (what it is) and operational definition (how it is measured)
what is a leap of faith?
assigning numbers to individuals in a systematic way, according to a rule/convention (imposing a scale)
imposes a relationship upon indicators that may/may not be justifiable
what are non-paramedic operations
not leap of faith
stat methods that don’t make assumptions (no number assignment)
what are the two main implications of number assignment (leap of faith)
people will be given a score than can be compared
you will get a distribution of scores from the group
what is a distribution of scores?
visual representation of number of each score
what is central tendency?
central/middle part of a distribution of scores
what is the median score?
absolute middle of a group of scores
can be smaller/larger than the mean
easily influenced by extreme scores
what is standard deviation
a single number than can describe the degree to which very person in a group tends to deviate/vary/spread out
allow you to compare groups of people with a single number
what does the ability to compare individuals depend on?
amount of test scores variability
there needs to be enough variability in scores to capture the actual degree in variability among test participants for proper comparison
variability allows you to see how characteristics are related
what is a score
when constructs are represented by numbers
what is a variable
what a score is referred to as when used in analysis
what are the three main types of variables?
discrete, continuous, and dichotomous
what are discrete variables?
difference between one member of group and next is very discernible (ex. 1% and 80%)
usually a very small # of subgroups
ex. race, year of study
what are continuous variables?
differences between one member of group and next are not very discernible (ex. 78% and 79%)
many different possible values (ie. 0 to 100% on a test)
ex. severity of depression, midterm grade, etc.
what are dichotomous variables?
discrete variables that have two levels
ex. gender used to be
what is the benefit of expanding discrete categories?
allows for examination of differences among individuals that were ignored before
ex. measuring gender as only female/male not representative and could be expanded
when are continuous variables converted to discrete?
when slight differences (ex. 78 to 79%) are less important than the broader category (B+)
or
made when the construct being studied is more discrete in nature
what is a true score?
most accurate representation of the actual amount of ability/traits/condition/knowledge
score you would achieve if conditions were completely ideal
very difficult to achieve, but possible
what are the two main types of variation?
unsystematic and systematic variation
what is unsystematic variation?
variation in test scores that can be attributed to unpredictable and random events affecting individuals taking the test
events are considered random because they don’t affect everyone
can work out in your favor, or harm you
ex. guessing answers, being tired
what is systematic variation?
variation in scores that can be attributed to differences among individuals (or subgroups of individuals) that are predictable
is something that affects everyone, or a large subgroup of people taking the test
ex. bad prof, bus not showing up for majority of class
is the greatest amount of variation among scores attributed to systematic or unsystematic differences?
systematic differences
what would happen in terms of variation if a test was too susceptible to random factors?
there would be too much unsystematic variation
scores would not be stable/reliable
different results would be yielded every time
what would happen in terms of variation if a test was not influenced by random factors
test would produce only systematic variation
scores would be very stable/reliable
scores would be the same every time
what is the main goal of tests in terms of variation?
to produce scores that vary more because of systematic variation rather than unsystematic variation
tests that have more unsystematic variation will not be as reliable
what is the difficult involved with multiple systematic effects?
difficulty arises when a questionnaire was designed to measure one construct (being influenced by one systematic effect) gets influenced by 2+ systematic effects instead
what is systematic bias?
a systematic effect
doesn’t happen at random
puts part of group at disadvantage
causes systematic variation in test scores
what are the concerns of systematic bias?
measurement concern = because scores no longer determined by what you were intending to measure
equity concern = because one group is at a disadvantage
what are the 3 sources of variation effects?
systematic effect (of what is being measured
systematic bias (that effects individuals in one or more groups)
unsystematic random error/bias (error that can affect individuals test scores at any time)
