Unit 4+5 Flashcards
chapter 4, 5, 6, 7, 12
what do observational designs consist of?
researcher observing and systematically recording some particular behaviour for the purpose of better describing the behaviour in question
ex. how children interact at a playground
ex. how rats at a dump share food
ex. how sales influence shopping behaviour
ex. how females interact with males in a bar
what are the two primary categories of observational research?
direct observation methods
indirect observation methods
what are direct observation methods?
ex. people watching is direct observation however to make it scientific you must record data
what are the two ways to directly observe?
without intervention
with intervention
what is the difference between without intervention and with intervention?
with no intervention researchers interfere as little as possible with a participants behaviour and direct observation with intervention is when the researcher purposely alters some aspect of the environment
what is a common type of direct observation without intervention?
naturalistic observation
what is naturalistic observation/ field observation?
normally is a qualitative method based on systematic observations that involves observing subjects in their natural environment
sometimes a mixed method of quantitative and qualitative is used as well as quantitative
field notes are created and can include information about all aspects (ex. setting, patterns of personal relationships…)
informed consent is usually unnecessary
what are limits to naturalistic observation?
cant be used to study all issues or phenomena
difficult to conduct (weird times)
interpreting the data is not simple
what are the two key issues with naturalistic observation?
whether to actively participate in the situation you are studying or not participate and only observe
whether to conceal your purpose or presence from the other people in the setting (concealed observation)
what is a common type of direct observation with intervention?
participant observer (they will have an active insider role)
what is participant observer research?
the researcher is the participant and the observer roles. this type of research can be disguised (people dont know their behaviour is being recorded) or undisguised (people know that their behaviours are being watched and recorded)
how do participants react to concealed observation?
results in less participant reactivity
whether to conceal or not depends on ethical and on the nature of the the particular group being studied
they normally will not disclose their purpose of the experiment
disguised participant observer research (concealed observation) minimizes the problem of reactivity, what is reactivity?
principle that states that participants or subjects respond differently when they know they are being observed
also called the hawthorne effect (because reactivity was first noticed in the hawthorne plant)
how can we minimize reactivity?
concealed observation
nonreactive or unobstructive operationalizations
allow time for people to become used to the presence of the observer and any recording equipment
what is participant reactivity?
the presence of the observer can affect peoples behaviours to cover this up we can use concealed observation (ex. one way mirrors, hidden cameras)
what is systematic observation?
the careful observation of one or more specific behaviours in a particular setting (setting is often created by the researcher)
used more often with quantitative approach
use a coding scheme often
what is a coding scheme?
a simple system researchers use to record, categorize and quantify qualitative observations to fit the needs of the particular study
ex. conversations coded for (1) topic, social or research related and (2) gender of the person they were speaking to. this study found that faculty were less likely to discuss research with female colleagues
sometimes researchers use coding schemes that already exist, what is the facial action coding system?
way to categorize subtle, fleeting, facial muscle movements
sometimes researchers use coding schemes that already exist, what is the mealtime interaction coding system?
codes interactions of family members during mealtimes
what is an independent variable and dependent variable?
independent variable is considered to be the cause (the independent variable causes changes in the dependent variable, we manipulate independent variables)
dependent variable is considered to be the effect (participants response to the manipulated variable)
what are the three requirements of cause and effect?
temporal precedence (causal variable must come first in the temporal order of events, then followed by the effect)
covariation between the two variables (changes in one variable must be accompanied by changes in the other)
eliminate plausible alternative explanations for the observed relationship
(if an experiment has all three of these, it is considered to have high validity)
how does the experimental method rule out alternative explanations?
by using random assignment (ex. using a list randomizer and large sample)
different forms of experimental control
what is a field experiment?
not the same as field observation (because field observation involves intervention). in field experiments the independent variable is manipulated in a natural setting out in the real world (ex. shopping mall, street corners)
what are advantages and disadvantages to field experiments?
advantage is that it takes place in a natural context, disadvantage is that the researcher loses the ability to directly control many aspects of the situation
what is a case study?
provides a detailed description of an individual person (ex. clinical psychologist) or setting (ex. neighbourhood, school, business)
when is the case study method useful?
when an individual possesses a particularly rare condition, in these cases studying more than one person might not even be possible
ex. a man in a test study was able to memorize long passages super easy
ex. the study on the twins were ones penis got cut off by accident and was raised as a girl
what is psychobiography?
a researcher applies psychological theory to explain the life of an individual usually in important historical figure
what is a causal claim?
a claim of the form “A was a cause of B”
what is the difference between an experimental method and a non- experimental method?
non- experimental: relationships are studied by measuring or observing the all the variables of interest while experimental methods involve direct manipulation of one variable, control of several other variables and measurement of the outcome variable.
how would a researcher study the relationship between exercise and happiness using a non- experimental method?
ask people to report their exercise habits and current happiness
why are both the non- experimental and experimental research methods necessary?
to fully understand behaviour
what is the best method to use, experimental or non- experimental?
a mixture of both (the strengths of one method may be able to compensate for the weaknesses in another)
what is the downfall of the non- experimental method?
cannot tell us whether two things are causally related. there are 3 options, A might cause B, B might cause A or there might be a third variable C (this is the third variable problem)
ex. exercise causes increased happiness, happiness causes increased exercise, higher income results in more exercise and increased happiness
what is a confounding variable?
variables that we are not interested in but are intertwined with our variable of interest
what is the difference between a third variable (lurking variable) and a confounding variable?
lurking variables are not included in the study, whereas confounding variables are included in the study, but their effects are not properly accounted for
what can help to address the third variable problem?
partial correlation (compare calculated partial correlation with original correlation
what is structural equation modelling (SEM)?
examines models that specify a set of relationshups among many variables represented in a path diagram
based on correlation so cant determine a causal association
waht are inferential statistics?
make inferences about the population from which the sample was drawn using sample data
what are three different relationships for a variable?
positive linear relationship
negative linear relationship
curvilinear relationship
what is a positive linear relationship?
an increase in one variable causes an increase in the other
ex. narcissism increases with increased time spent on facebook
what is a negative linear relationship?
increase in one variable causes a decrease in the other
ex. as satisfaction with a therapist increased, the number of students with depressive symptoms decreased
what are two things to look for when looking at correlation (r)?
restriction of range (important that the researcher sample from the full range of possible scores)
curvilinear relationship (r coefficient cant detect these relationships, only linear relationships)
what is the squared correlation coefficient? why do we use it?
r squared
to get the proportion of variance being explained
what is a curvilinear relationship?
increases in one variable cause increases and decreases in another variable
ex. teachers report that they are effective at engaging their students (increase) but eventually it leads to a decrease. (this would be shown as an inverted U relationship)
what is a situational variable?
describes characteristics of a situation or environment to which the participant is exposed (ex. number of people around you while writing a test)
measured in any design
what is a response variable?
responses or behaviours of participants, such as reaction time in response to a stimulus, performance on a task and emotional reactions
measured in experimental or non- experimental
what is a participant variable?
describes a characteristic that individuals bring with them to a study (cultural background, age, intelligence, personality traits)
what is more abstract, a hypothesis or a prediction?
hypothesis
when are self report measures used?
to study personality traits, attitudes, evaluations, motivations and preferences
what are operationalizations?
variables must be translated into concrete and specific forms (reliability and validity are important for operationalizations)
ex. aggression must be defined in terms of the specific method that will be used to measure or manipulate it
ex. intelligence might be operationalized as academic average
what is the correlational method?
methods in which variables are observed but not manipulated (NOT the same as the correlational statistic)
what are two types of indirect observational methods?
archival research
physical traces
what is archival research? what types of information is contained with archival records?
documents (or records) that can be found at any university libraries, government buildings and other institutions
information in archival records may include: data on birth weights and rates, death rates and causes of death, weather reports and crime rates
what are three sources of archival research?
census data in the form of statistical records (ex. stat canada)
survey archives
written records and mass media
what are written records?
stored documents (diaries and letters)
what are mass media records?
books
magazine articles
movies
television programs
newspapers
websites
what are some problems with archival research?
bias can creep into findings from selective deposit (occurs when archival research is established and data could have been altered in some biased fashion) and selective survival (how archives survive over time)
archival research involves content analysis, what is it?
systematic analysis of existing archives such as written documents and mass media records like movies and television shows
what are the 3 challenges associated with the use of archival data?
the desired records may be difficult to obtain (data could be owned by companies for ex.)
cant control what data were collected or how they were recorded
its non experimental so we cannot make causal claims about these associations
how do you avoid bias in content analysis?
classification and evaluation of the archival data should have two people or judges trained on the methods to be used in data classification and evaluation, both judges then code the archival data independently, the findings are then compared which gives an indication of reliability, reliability is high if the observers record the same or similar results on given behaviours, if reliability is high then there is little bias in the archival data
what are physical traces?
physical evidence left behind by people or animals to study behaviour
ex. graffiti in school bathrooms
ex. garbage people throw away
ex. bear dung or scratches on a tree
what is a population?
a set of people of interest to the researcher (ex. eligable
what is a sample?
group that is selected to represent the population
what is a confidence interval?
a way to quantify uncertainty that surrounds an estimate
gives information about how much error likely exists
why is a larger sample size better?
reduces measurement error
reduces the size of the confidence interval
what is a probability sample?
each member of the population has a known and specific probability of being chosen
what are the 3 methods of probability sampling?
simple random sampling
stratified random sampling
cluster sampling
what is simple random sampling?
if every member of the population has an equal chance of being selected into the sample and every possible combination of those members also has an equal chance
consider that the entire student body at U of M is our
population. Which of the following three procedures best describes a simple random sample?
Dr. X’s sample includes 100 students from her Distance Education class and 40 students from
her on-campus class.
Dr. Y goes to the year-end Commerce party and samples 200 people.
Dr. Z puts each student’s name in a big hat (30,000 student’s names in the hat), mixes them
with her hand, and draws 150 names from the hat.
Dr. Z
how might simple random sampling not be random
because it might not be very representative of the population as there may not be much variation
ex. out of the 150 names in dr. Z hat 120 of them may have been male and only 30 female (not so random) to deal with this problem we perform a stratified random sample
what is a stratified random sample?
researcher divides the population into subpopulations (strata) and then draws a simple random sample from each strata
ex. divide u of m population by males and females and take a random sample from each of the strata
what is cluster sampling or multistage sampling?
used when you know very little about the population
identify clusters of individuals and take a simple random sample of these clusters, if they take a sample of that cluster thats when it becomes multistage
ex. researcher wants to study the political attitudes of canadians, first they might divide canada by postal code and select a simple random sample of postal codes, next they might divide selected postal codes by street names and select a random sample of streets, next they might select houses on the street
how do researchers achieve representative samples of behaviour in observational research?
they use a combination of time sampling, situation sampling and non- probability sampling
what is time sampling?
making observations of behaviour at different time periods in order to obtain more representative samples of the behaviour of interest (time periods are chosen either systematically or in a random fashion)
what is situation sampling?
observing the same behaviour in many different situations (increases ability to generalize) in order to obtain a more representative sample (allows researchers to determine if the behaviour of interest changes as a function of the environmental context)
what is a non- probability sample?
dont know the probability of any particular member of the population being chosen
what are implications about non probability sampling?
generalizability of any results based on the sample
what are the 3 types of non probability sampling?
convenience sampling/ haphazard sampling
purposive sampling
quota sampling
what is haphazard sampling?
POPULAR SAMPLING TECHNIQUE where participants are recruited wherever you can find them. presents bias (ex. doesnt represent all students at u of m)
ex. stand outside a building at school and ask people that walk by your question (people that dont walk by this building wont have a chance to answer the question)
what is purposive sampling?
sampling for a specific purpose to obtain a sample of people who meet some predetermined criterion
ex. instead of sampling every person who walks by, only ask people who are men under age 30 the question as they walk by
what is quota sampling?
similar to stratified sampling but without randomness
ex. you want your sample to be 19% first years, 23% second years, 26% third years, 22% fourth years and 10% graduate students. quota sampling would ensure you have these percentages but would collect data using convenience sampling techniques
what is a measurment?
process of assigning numbers and thus the properties of the number system to events according to a predetermined set of rules
involves two parts: the rules or the procedure that are used and the scale that is created
what are 4 types of scales?
nominal scales
ordinal scales
interval scales
ratio scales
what are nominal scales?
the most basic scale that uses simple qualitative (no numeric values) differences in the variables being measured thus, numbers are assigned to groups or categories (the only numbers in this scale type are labels that refer to people or data NOT numeric calculations)
ex. study of what countries people are from
ex. two kinds of therapies for depression
Appropriate statistics: Frequencies and chi-square
what is an ordinal scale?
allows us to order the levels of the variable in terms of a rank
ex. top ten list
Appropriate statistics: percentiles, correlation, Mann-Whitney U test
what do ordinal scales indicate?
an individuals or objects value based on its relationship to others in the group as the numbers in this scale type have a meaning within the group
ex. gold, silver, bronze (gold silver and bronze provides no information about the group)
Appropriate statistics: percentiles, correlation, Mann-Whitney U test
what is an interval scale?
the difference between the numbers on the scale are equal in size
we cant use ratios with intervals, for example, we cant say 20 C is twice as much as 10 C because temperature can also be recorded in fahrenheit and 68 F (20 C) is not twice as warm as 50 F (10 C)
ex. IQ scores, temperature in celsius
ex. difference between 90 and 95 is the same as 115 to 120
how is an interval scale an ordinal scale and nominal scale combined?
because it has magnitude (ordinal scale) but we also know exactly how much larger or smaller things are (nominal)
ex. if you score 80/100 on a test and bob scores 12/100 on the same test, you scored 68 points greater than bobs score
what does it mean that interval scales are assigned an arbitrary zero (true zero)?
ex. when its 0 degrees celsius it doesnt mean that theres no temperature outside
ex. if you score 80 on a math test and bob scores 20 on the same math test, it doesnt mean that you are four times as good as bob at math
ex. rating scale of extraversion from 1 to 7, we cannot say that the person who scored 6 is twice as extraverted as teh person who scored 3 because there is no absolute zero that indicates an absence of a trait
what are ratio scales?
a combination of interval scales, ordinal scales and nominal scales, except there is a true zero, comparisons can be made
ex. if nate steps on the scale weighs 90kg and you weigh 30kg, nate is 3 times my weight
Appropriate statistics: mean, median, mode, standard deviation, correlation, ratios, t-test, ANOVA
what is a frequency distribution?
indicates the number of participants who receive or select each possible score on a variable
can be created for variables using any scale of measurement
what are the are different types of graphs that are used to depict frequency distributions?
bar graphs
pie charts
histograms/ normal distribution
frequency polygons
what are bar graphs used for?
for nominal categories (not continuous) comparing group means, percentages
ex. individuals choose one of five responses to the given question “ which of the following problems do you consider to be the most serious?”. the x axis would be the 5 options and the y axis would be the percentage of people.
when are pie charts useful?
when representing data on a nominal scale
can only be used if the sum of the individual parts add up to a meaningful whole
used in applied research reports, newspapers, magazines
what are histograms?
bars that all touch each other (because it uses a continuous variable) normally in the form of a bell curve
when are histograms used?
used for a continuous variable. values along the x axis are continuous and show increasing amounts of a variable (ex. blood pressure, number of correct responses)
what is an example of a histogram?
normal distribution: majority of scores cluster around the mean,
what is a frequency polygon?
an alternative to histograms
helpful when you want to examine frequencies for multiple groups simultaneously
(looks like a line graph, with a line for each value being measured)
ex. when looking at the age of respondent from mexico compared to australia, mexico would be orange and australia would be blue for ex
what is a descriptive statistic?
when you calculate statistics to describe or summarize data
what are two main types of descriptive statistics?
measures of central tendency
measures of variability
what is a measure of central tendency?
a statistic that tells you what the scores are like as a whole or how people scored on average
what are the 3 measures of central tendency?
mean
median
mode
when is mean used? what scale
when wanting to find the average
when using interval or ratio scale
very susceptible to outliers
when is median used? what scale?
ordinal scale
continuous
what is mode?
most frequent score
when is mode used? what scale?
when deciding what is most popular
interval
ratio
ordinal
what is a measure of variability?
amount of spread in a distribution of scores for continuous variables (interval or ratio scales)
what are two common measures of variability?
standard deviation (small when values are close to the mean) (most common)
range
what are the two types of data analysis?
quantitative (when data varies in numerical amount, requires statistical analysis and numerical data)
qualitative (cannot be numerically manipulated (ex. single, married, divorced))
how can interobserver reliability be calculated?
(number of times two observers agree)/ (number of opportunities to agree) x 100
can qualitative research be summarized using statistics?
NO (qualitative research conclusions are based on investigative interpretation of the data and very little statistical calculations are involved
what are the two different ways to describe how qualitative variables (nominal data) are related?
comparing group percentages
comparing group means/ averages
what is a common way to graph nominal data?
bar graph
line graph
what is something to watch out for with bar graphs?
look at the y axis to see what the values are. many politicians trick readers by making their y axis appear more dramatic than it should (ex. making y axis 7 to 9 instead of 1-9)
what is effect size?
a general term for size, amount or strength and helps us interpret how large our effects are
when comparing two groups on their responses to a continuous variable what is an appropriate effect size?
cohens d (difference in means between two groups standardized by expressing it in units of standard deviation)
d of 1 indicates that the means are 1 standard deviation apart
d of 0.2 indicates that means are separated by 0.2 standard deviations
smallest d value is 0 which indicates no effect
what is a mediating variable?
psychological process that helps to explain the relationship between two other variables
what is the pearson correlation coefficient?
a measure of the relationship or association between two quantitative variables measured on the same person or thing using a range from -1 to +1 (r= 0 means the variables are unrelated)
what are the two properties of correlation?
direction of the correlation (positive or negative)
strength of the relationship
how do we determine the direction of correlation?
using scatterplots.
positive relationship is when both variables are increasing, negative relationship is when one variable is increasing and the other is decreasing, if theres no specific pattern then the relationship is zero
waht is good about scatterplots?
allow us to detect outliers (outliers may be data entry error or it might be a real outlier point)
how do we know the strength of the relationship?
by using regression equation (Y= a+bX), the closer r is to -1 or +1 the stronger the relationship. it summarizes all the data points
ex. r= -0.95 is a stronger relationship than r= +0.94
what is multiple correlation?
represented as R
correlation between a combined set of predictor variables and a single criterion variable
generates a stronger relation
what is multiple regression?
examines unique relationships between each predictor and the criterion
accommodates more than one predictor to predict the criterion variable
when do researchers use multiple regression?
to study basic research topics
what is a limitation of correlation?
- dont allow us to draw cause- and- effect conclusions about the studied variables (even if variables are highly correlated they might not be causally related)
ex. male prisoners that watch porn are highly correlated (r= +0.8) to violence towards women, but there may be a third variable like childhood abuse that is making them correlate - which comes first?
ex. we couldn’t say that prisoners violent attitudes towards women caused these men to view pornography - participants are not randomly assigned to an experimental or control condition
what is a spurious relationship?
make it clear that correlation cannot imply causation
ex. sales of ice cream and the number of shark attacks on swimmers are correlated
what are surveys often used for?
used to gather large amounts of data about information people provide about themselves
what are two forms of surveys?
questionnaires (personal administration to groups or individuals, mail surveys, internet surveys)
interviews (face to face, telephone, focus group)
what is a focus group?
a group of 6- 10 people is gathered, open ended questions are asked and everyone must respond, its costly (need an incentive for ppl to show up), time consuming, provides lots of info on one topic
what are advantages and disadvantages to questionnaires?
advantages: low cost (can be online surveys), allow respondent to be anonymous,
disadvantages: respondents must be attentive enough to complete them honestly, low response rate in mail surveys
advantages and disadvantages to interviews
advantages: higher response rates, interviewer can clarify questions for the respondent,
disadvantages: interviewer bias (interviewer influence the respondents answers),
what are tests and inventories often used for?
to measure some specific characteristic or trait possessed by an individual
why are surveys and questionnaires popular for psychologists?
easy to conduct
can gather lots of data in a short amt of time
versatile (can gather info on any topic)
can provide a snapshot of how a particular group of people think and behave at a given point in time
used for experimental research
what is a panel study/ longitudinal study?
when the same people are tracked and surveyed at two or more points in time
what is the downfall to survey research?
response bias (people need to provide truthful and accurate answers)
what is a response set?
a tendency to respond to all questions from a particular perspective or in a particular way rather than provide answers directly related to the questions themselves
what are the two types of surveys and questionnaires?
descriptive surveys
analytical surveys
what are descriptive surveys?
provide nominal data that allows you to determine the percentage of the population that holds a certain view
what are analytical surveys?
generate nominal, ordinal and interval data
descriptive and try to determine how particular variables are related to one another
what are the 6 steps to build a good questionnaire, survey, test or inventory?
determine the information you need and how you will get it
determine the style of questions you should use
create the specific items to be included in the survey or questionnaire
pilot the survey or test items and get a second opinion
determine the relevant demographic variables
write instructions and administration procedures
what are the three general types of content measured by survey questions?
attitudes and beliefs (how you feel about issues)
facts and demographics (indicate things they know to be true about themselves and their situation)
behaviours (things you have done in the past or plan to do in future)
what are the two types of questions you could use for a good questionnaire, survey, test or inventory?
open- ended (answers will be in their own words; constitute qualitative data)
closed- ended (answers are limited)
what are different types of close- ended questions?
yes- no questions (statistically: frequency or percentages)
forced choice or forced alternate questions (force participants to choose between two options)
multiple choice questions
likert scale questions (rating scales, measure the magnitude of an opinion) can calcualte the mean for the data
when using scales for questionnaires theres scales that are fully labelled, and scales that only have the lowest option (1) and highest option (5) labelled, what is the issue with these?
fully labelled is better however, sometimes people use the middle of the scale if they dont know how to respond, so it is recommended to also have an “idk box”
what is a graphic rating scale?
when respondents need to make a mark along a line that has descriptions at each end (lowest and highest ends)
what is a semantic differential scale?
way to measure the meaning that people ascribe to concepts, respondents rate concepts based on a continuum spanning two adjectives (one at each end, ex. bad ——–good)
semantic differential scales are rated along 3 basic dimensions, what are they?
evaluation
activity
potency
what is a non- verbal scale?
only using diagrams (used for kids)
disadvantages of open- ended questions?
too broad or too simplistic answers
answers are hard to deal with statistically bc you have to decode their answers
disadvantages of closed- ended questions?
difficult to create the questions
participants might not agree with any options you give (miss out on data)
topic youre studying may not be able to be reduced to a few choices no matter how thought out the choices are
what are some guidelines to follow when writing the survey or questionnaire? step 3
attempt to address only a single issue per question
use vocabulary appropriate for participants (simple)
put more interesting questions at the beginning
avoid bias (loaded questions)
avoid negative wording
for mc make all your alternatives clear
what is a double barrelled question?
a question that asks two things at once (AVOID THIS)
what is inattentive responding?
when people provide answers without even really thinking about them. someone who is not paying attention may just agree to all the questions (called yea saying) or may disagree to all the questions (called nay saying)
to solve this, the person asking questions could ask a question that requests a particular response (for this question answer is d)
what is social desirability?
a type of response set
its when an individual answers the way they think “most people” would respond
what does piloting the test or survey mean?
giving it to a small group of participants and asking for feedback about the survey or test after getting feedback you can change some of the options to questions or change questions entirely
also get someone in the psychological field to look at it to ensure you and the participants didnt miss anything
what are demographic variables?
questions about the characteristics of the participants that will take your survey
ex. their annual income, gender, age, height…
what are some things to keep in mind when writing instructions for the survey (or test) participants?
be specific and concise
put instructions at beginning of survey
indicate how participants should respond to questions (circle the answer, highlight), encourage honest answering
what are some things to keep in mind when writing instructions for the survey (or test) administrators?
prep a script for the administrator to read to the participants
tell administrator how many people will be taking the survey
be specific and concise with instructions
required equipment (ex. privacy needs, quiet, chairs, tables, desks…)
specify time limitations
why do we go through so much trouble developing a survey or test?
to ensure reliability and validity
reliability is super important for measurement. any measurement includes two components, what are they?
a true score (the persons actual level of the variable of interest(not the score they get on the measure of that variable)
measurement error (any contributor to a measures score that is not based on the actual level of the variable of interest)
ex. for a reactivity test if you had to press a certain key when you hear a certain sound, if you press the wrong key then press the right key that would be measurement error
what is reliability?
consistency of a measure (if highly reliable you should get around the same result)
ex. if you were to give the survey to the same people for a second time, all the answers should be the same, if the answers are not the same then the test is not reliable or accurate
what are the 4 main strategies to measure reliability?
test- retest reliability
alternate forms reliability
internal consistency reliability (type of internal consistency reliability is split half reliability)
inter- rater reliability
what is test- retest reliability?
when a survey developer gives the same survey to the same group of participants on two different occasions (could be in a couple years or hours) scores from each test are then compared (survey is reliable if correlation is large and positive)
what would you expect as the time interval between test administration increases?
decrease in reliability because people change drastically over long periods of time
practice or carry- over effects are two concerns of test- retest reliability, what is it?
the benefits participants obtain from already having seen a survey or questionnaire
ex. get given a test, in a couple days you retest, your scores will be better the second time around
to minimize carry over effects of reliability, what is an alternative reliability method we could use?
alternate forms reliability
what is alternate forms reliability?
when survey developers produce two highly similar forms of the test, then correlate the scores on one test with the scores on the other test to give a measure of reliability
does not eliminate the carry over effect because participants will remember the types of questions that were asked the first time, thus we use consistency reliability
what is internal consistency reliability?
examines how successful the different items in a scale are at measuring the same construct or variable (when people respond similarly across these different attempts it suggests that the measure is reliable)
measure the relationship (correlation) between each and every individual item on any survey or questionnaire
why is internal consistency reliability such a good method?
because it eliminates or reduces:
- the need for two administrations of your survey
- the difficulty of developing another form
- carry over and practice effects
- changes that may occur in your participants over time
what is cronbachs alpha?
a statistic used as an indicator for internal consistency
each item on a test or survey is compared to every other item on the test or survey, this produces inter- item correlations the average of the inter- item correlations gives the value of cronbachs alpha (values of cronbachs alpha that are higher than 0.7 suggest that your test is reliable)
what is split half reliability?
a form of internal consistency
give a survey or questionnaire to a group of participants, once they have taken the survey randomly split the survey in half so you essentially have two surveys. compare the two surveys for each participant and estimate the reliability of the test
what is sampling error?
differences between what is present in a population and what is present in a sample
why is split half reliability not as reliable?
because when you divide a survey it decreases its reliability
what formula can be used to estimate split half reliability and how much a surveys reliability will increase or decrease?
The spearman- brown formula
what is inter- rater reliability?
raters observe behaviours and make ratings or judgments. high inter- rater reliability is obtained when most of the observations for two or more raters results in the same judgment
what is validity?
when a test actually measures what it says it is supposed to measure
ex. if you want to measure fear of heights, you cant just weigh the person and say thats a measure of fear those results would not be valid
a study has high validity when it is well designed
what is external validity?
the extent to which results can be generalized from the sample to the population from which it is drawn (1. randomly sample from a pop. that contains all people of interest, 2. contact selected people)
what is sampling frame?
actual population of people (or clusters of people) from which a random sample will be drawn (often a subset of the population of interest)
ex. if you define your population of interest as “residence of saskatoon” the sampling frame may be based on a list of home telephone numbers that you will use (not everyone has a home phone)
what is a weakness of the split half method?
reliability can change depending on how you split the test (higher split is higher correlation value)
what are the 4 types of validity?
face validity
content validity
construct validity
criterion- related validity (or predictive and concurrent validity)
what is face validity?
only involves judgment, the measure appears on teh “face of it”
does not tell us about what a test actually measures and instead refers to how test takers perceive the attractiveness/ appropriateness of a test
what does it mean if a test is perceived not to have face validity?
test takers might hurry through a test and take it less seriously by undermining the usefulness of the test
what is content validity?
the more representative the tests content is to the thing that is being measured (ex. depression) the greater the content validity of the test
how would you know if a test has content validity?
- research the thing being measured (ex. research methods)
- through the research you discover that research methods are made up of 4 topic areas, each topic area weighs differently (more or less importance)
- you discover that topic 1 is 40% importance, topic 2 is 30% importance, topic 3 is 20% importance and topic 4 is 10% importance
- within the test if 40% of the test is topic 1, 30% is topic 2, 20% is topic 3 and 10% is topic 4 than we can be sure that the test demonstrates content validity
what does face validity and content validity have in common?
both focus on assessing whether the content of a measure reflects the meaning of the construct being measured
what is construct validity?
extent to which a test measures some theoretical construct (construct is something that exists in the theoretical sense) ex. we cannot see shyness
what are the two ways of gathering evidence for construct validity of a test?
theoretical evidence
psychometric or statistical evidence
what is theoretical evidence?
proposes an experimental hypothesis using the test as a measure of the construct in question, if the test is a valid measure of the construct then test scores should support the experimental hypothesis
what is convergent validity?
when a test has construct validity then it should produce scores that correlate strongly with other tests that measure the same construct
high correlations
what is discriminant validity?
opposite of convergent validity
when a test has construct validity and does not correlate well with other test scores that measure an unrelated construct (ex. r=0.09)
what is criterion- related validity?
if the test correlates well with seperate, independent behaviours that are associated with the construct then it is said to have criterion- related validity
when is criterion validity important?
when studying measures designed to improve our ability to make predictions about different behaviours
what are the two main methods used to determine the criterion- related validity of a test?
predictive method
concurrent method
what is the predictive method?
give the test to two large groups of people, their scores are held for a predetermined amt of time, when the time is done the same people are measured on specific behaviours (predicted by the test), test scores are then correlated with criterion scores (if correlate + then you. have criterion related validity
what is the concurrent method?
similar to predictive method but test scores and criterion measures are gathered at roughly the same time (used most often for criterion- related validity)
