research methods exam 2 Flashcards
self report measurement
type of measurement
-participants provide information about themselves through a survey, questionnaire, interviews or even diaries giving responses to pre-set questions
Ex: Likert scale
observational/behavioral measurement
type of measurement
-researcher observes & records some behavior
Ex: kids in class, rats in lab
physiological measurement
type of measurement
-recording any of a wide variety of physiological processes
Ex: heart rate, pulse, blood pressure, eye tracker
categorical variable
scale of measurement
-countable number of distinct groups based on a characteristic
-cannot be ordered or measured numerically
Ex: ordinal
quantitative variable
scale of measurement
-represents data in numerical form and these numbers have meaning attached
Ex: ratio, interval
ordinal measurement
scale of measurement
-represents categories and groups with a ranking order
-first, second, third
-unequal distance between categories
-no true zero
Ex: education levels, stages of disease, sports ranking, grades (A-F)
interval measurement
scale of measurement
-numbers ordered on a scale
-intervals between data points are consistent and meaningful
-no true zero
Ex: temperature, IQ, standardized test scores
ratio measurement
scale of measurement
-data on a scale with a true zero
-equal intervals
Ex: height, weight, age, time, distance, test score (0-100), exam (number of correct answers)
face and content validity
-does it look like a good measure?
-both face and content validity are subjective ways to assess validity
face: it looks like what you want to measure
content: the measure contains all the parts that your theory says it should contain
criterion validity
-does the measure correlate with key behaviors?
-correlational evidence for criterion validity
Ex: NCLEX correlates with being a good nurse
Ex: SAT doesn’t have good criterion validity - it is not directly tied to good/bad results or success in school
convergent & discriminant (divergent) validity
-does the pattern make sense?
convergent: test of the extent to which a self-report measure correlates with other measures of a theoretically similar construct
discriminant: test of the extent to which a self-report measure does not
correlate strongly with measures of theoretically dissimilar constructs
test-retest
consistent scores every time the measure is used
interrater reliability
consistent scores no matter who does the measuring
internal reliability
internal consistency; participants provide a consistent pattern of responses, regardless of how the researcher has phrased the question
open ended survey question
-allows respondents to answer any way they like
more detail, but more time
forced choice survey question
-forces people to choose yes/no or pick the best of 2 or more options
less time, clear answer, but less detail
Likert scale survey question
a rating scale containing multiple response options anchored by the specific terms including term “(dis)agree” (strongly agree-strongly disagree)
Likert-type scale
basically the same as Likert, but not using the term “(dis)agree”
-can be likeliness, satisfaction, importance
semantic differential format survey question
response scale whose numbers are anchored at both ends with contrasting adjectives
Ex: horrible 1 2 3 4 5 fantastic
wording questions
-more variations in question wording than type of question
leading questions - questions/wording to avoid
-wording encourages one response more than others
-weak construct validity
keywords: “do you agree?”, “wasn’t the…
?”
double-barreled question - questions/wording to avoid
-asks 2 questions in 1
-weak construct validity
-confuses people on which question to answer
negatively worded questions - questions/wording to avoid
-contains negatively phrased statements, making its wording complicated or confusing
-potentially weakens construct validity
question order
-order in which questions are asked can also affect the responses to a survey
-to fix this: randomize the question order & prepare different versions of a survey with the questions in different sequences
response sets (aka nondifferentiation)
-always answering the same, not fully reading each question
-happens in long surveys, people want to finish it
-answer consistently with positive, negative or neutral
-to fix this: reverse the question wording
acquiescence (yea-saying)
-agree with all questions, again without truly reading questions (“strongly agree” or “yes”)
-to fix this: reverse wording of questions
fence sitting
-consistent answer in the middle, not choosing a side
-occur when people think answers might be controversial or question is confusing/unclear
-to fix this: use forced choice questions & even number scales by removing the neutral/middle option
faking good (socially desirable responding)
-giving answers on a survey that make people look better than they really are
-people want to look good in the eyes of others & don’t want to give an unpopular opinion
-to fix this: ensure participants of anonymity of responses, ask people’s friends to rate them (because others know us better in domains where we want to look good)
-to fix this: use computerized measures to evaluate people’s implicit opinions about sensitive topics
faking bad
-giving answers that make one look worse than they really are
self-reporting more than they can know
more than they can know:
-self-reports can be inaccurate, especially when people are asked to describe why they are thinking, behaving, or feeling the way they do
-when asked, most people willingly provide an explanation/opinion to researchers, but
sometimes unintentionally give inaccurate responses
self-reporting memories of events
-memories for significant life experiences can be quite accurate, but other times memories might not match the accuracy
observation bias
-when observers see what they expect to see
-observers’ expectations influence their interpretation of the participants’ behavior or the outcome of the study
preventing observer bias & observer effects
-use a masked design: observers do not know to which conditions the participants have been assigned & they are not aware of what the study is about
observer effects
when participants confirm observer expectations
-expectancy effect vs reactivity
expectancy effect
observers inadvertently change the behavior of those they are observing
-participant behavior changes to match observer expectations
reactivity
participants reacting to being watched
-can react to be on their best behavior, their worst, or just behave differently
population
the entire set of people or things in which you are interested
sample
the smaller set of people/things taken from the population
-intended to serve as an estimation of the population
census
sampling every member of the population
-has the highest validity, because it represents every person
convenience sampling
biased method
-sampling only those who are easy to contact
-very common & easy
Ex: professor using students as participants
self-selection sampling
biased method
-sampling only those who volunteer to participate
Ex: rating something online, online quiz
simple random sampling
unbiased method
-obtained by putting every member’s name of your population of interest into a pool and then randomly selecting a predetermined number of names from the pool to include in your sample
-NOT random assignment
systematic sampling
unbiased method
-researchers select members of the population at a regular interval
Ex: I could roll two dice. Let’s say one lands on
two and the other lands on three. I would start
at the second person in the class and then
choose every third person until the sample
reached the desired size
simple random & systematic sampling
extremely time consuming
not always possible (cannot access population of interest)
very difficult
cluster sampling
unbiased method
-a probability sampling method in which you divide a population into clusters, such as districts or schools, and then randomly select some of these clusters as your sample
Ex: classes at Assumption, randomly survey 10 classes
multistage sampling
unbiased method
-a method where researchers draw a sample from a population using smaller and smaller groups at each stage
-this method is often used to collect data from a large, geographically spread group of people in national surveys, for example
-this can be used to break down larger clusters
Ex: all college students - schools - randomly select schools - randomly select classes within schools
stratified random sampling
unbiased method
-method of sampling that involves the division of a population into smaller subgroups known as strata
-strata are formed based on members’ shared characteristics
-sample represents same proportion of population
oversampling
unbiased method
-variation of stratified random sampling that selects respondents so that some groups make up a larger share of the survey sample than they do in the population
settling for nonprobability/biased sampling techniques
-a random sample is not always attainable
purposive sampling
biased method
-used when you want to study certain kinds of people, so you only recruit those types of participants
Ex: selecting experts or top students to give feedback
snowball sampling
biased method
-a variation of purposive sampling in which study participants are asked to recommend other participants for the study
quota sampling
biased method
-similar to stratified random sampling
-the researcher identifies subsets of the population and then sets a target number (quota) for each category in the sample
-then uses nonrandom sampling until the quotas are filled
interrogating external validity
in a frequency claim, external validity is a priority
-to evaluate claim, evaluate sampling technique
in nonprobability samples (in real world & research studies), external validity is a lower priority
