Final review Flashcards
Loaded questions
contain emotionally charged terms, forces reader to admit certain assumtions
Leading questions
information leads the respondent to answer in a particular way
Double barreled questions
asking two things in one; which makes it difficult for respondents to know which to answer
major decisions in questionnaire design
- content
- wording
- Sequence
- Layout
CONTENT Questionnaire design
What will be asked?
WORDING questionnaire design
How should each question be phrased?
SEQUENCE - questionnaire design
in what order should questions be presented?
LAYOUT - questionnaire design
What format will best serve research objectives?
Survey Development Steps
- first draft
- pilot study
- Modify
- Test small sample
- Formal study
Response Rate Formula
(Number of responses divided by the potential number of responses) x 100
Potential Responses
total number in a sample minus ineligible or undeliverable requests
Biased sample caused by
low response rates from questionnaire
Mail Survey Design
- low response rate
- low interviewer bias
- good for personal questions
- expensive
Internet Survey Design
- low response rate
- Low interviewer bias
- may get multiple responses form one person
- Good for personal questions
- inexpensive
Phone Survey Design
- higher response rate than mail or internet, but lower than in person interview
- Not as good as mail or internet for personal questions
- Low expense rate
Personal Interview Survey Design
- Highest response rate
- Socially acceptable bias high
- structured
- Not as good for personal questions
- Higher Expense rate
Low cost surveys
Internet and phone surveys
High cost surveys
mail and in person interviews
Low response surveys
Mail and internet surveys
High response surveys
Phone interviews and in person interviews
Surveys
studying differences between groups
Questionnaires
studying relationships between factors
Questionnaires use questions to
obtain information about the thoughts or behaviors of a large group of people
Questionnaire sampling
a smaller segment of the population is used and assumed to reflect the entire whole
Types of surveys
Mail surveys, internet surveys, telephone surveys, personal interviews
ANOVAS
Factorial designs of two or more independent variables
Factorial design
two or more studies in one
Factorial ANOVA study puprose
by testing more than one variable at a time, we can look at the interactive effects of independent variables
Why factorial designs ANOVAS
most independent variables in psychology interact with other independent variables
Main Effects ANOVAS
The effect of each of the independent variables on the dependent variable
Interaction ANOVA
the combined effect of two or more independent variables on the dependent variable
Interaction of factorial design
is more than just the sum of the main effects
Two factors
two independent variables
t-Test
test used to look at the differences between two groups on variables of interest
t
(difference between groups - expected difference between groups)/(standard error of difference between groups
Independent t
comparing two experimental conditions with different participants assigned to each condition
Within Subjects design
Each subject given both experimental and control condition
one sample t-test
compares the mean of one group to a fixed estimate
independent samples t-test
compares the means of two independent groups
paired samples t-test
compares the means of two related groups
Measures of central tendency
mode, median or mean
Mode
the most frequent value
mode used for
nominal data, named data
Median
the middle score in a data set
Median used for
ordinal data
Mean
the arithmetic average of all the scores
Mean used for
interval, ratio and scale data
Dispersion
measure of the variability or spread in the distribution of scores
range
highest value minus the lowest value
standard deviation
the average distance from the mean for all the scores
Scales of measurement
nominal
ordinal
interval
ratio/scale
Nominal
scores represent a particular characteristic but have no actual value
examples of nominal measurements
gender, eye color, hair color
Ordinal
scores indicate whether there is more or less of the variable, but not how much
examples of ordinal measurements
likert scales, hotel ratings
Interval
equal distances between scores correspond to equal size changes
examples of interval measurements
Feirinheit scale, dates
Ratio
interval scales that have an absolute zero
examples of ratio measurements
Kelvin temperature scale, reaction time, age, salary
Measurement options on SPSS
Nominal
Ordinal
Scale
Type 1 error
H sub 0 is true for the population but you rejected H sub 0 based on the sample
Type 2 error
H sub 0 is false for the population but you failed to reject H sub 0 based on the sample
Statistics
used to organize and summarize data and sometimes make predictions about the population
Significant effect
the result in question is unlikely to have occurred in the sample by chance
Confidence level of 95%
means there is only a 5% chance that the results were not caused by chance
Alpha level p
most commonly used confidence level
A scientific experiement
there are a series of steps to test a hypothesis
Steps of a scientific experiment
- Question
- existing research
- hypothesis
- test hypothesis
- analyze data
- report results
types of scientific experiments
Experimental
Quasi-experimental
Observational (non experimental)
Experimental design
the most powerful scientific experiment because it can show cause and effect
Cause and effect
only provable with experimental design
Observational scientific experiment
used when there is no way to control variables, such as outside the laboratory
Observational scientific experiment design
- identify all confounds
- data collection consistent:
environmental conditions, timing of data collection, data collection instruments - same procedures for each subject in the design
Quasi-Experiment
No manipulation of the independent variable, correlations measured
Quasi-experimental design
- all variables are observed and data collected
2. researchers examine the correlations between and among variables of interest
Quasi-experimental design examples
- survey experiments describing answers provided in a questionnaire
- Correlation experiments examining the relationship between two or more variables
- 5 Studies where you can’t randomly assign such as pancreatic cancer studies
Quasi-experimental study
similar to randomized control experiment, except there may be a process required in the control experiment that is missing or unable to be accomplished. Sometimes no control group or groups cannot be randomly assigned
Experimental
randomized control, highest level of scientific experiment because there is the most amount of control and the only type of design to show cause and effect
Experimental design
at least two groups made up of subjects that resemble each other as closely as possible. Can be human, animal, plants or ecosystems.
Random sample (experimental design)
the subjects are randomly assigned to either the control or experimental condition
Random number generator
easiest way to randomize participants for a study
Experimental design format
everything is the same for both groups except for the independent variable, which changes in the experimental group only
Cause and effect relationship measured by
the differences between experimental and control group to see if there was any effect on the dependent variable from the independent variable that was manipulated
types of scientific experiements
experimental
quasi-experimental
observational
Types of within subjects designs
Pretest-post test
Repeated measures
Longitudinal Designs
Pre-test post test within subjects design
Independent variable is measured before and after some treatment or manipulation
Repeated-measures within subjects design
Independent variable measured on a number of occasions, but not necessarily following a treatment or manipulation
Longitudinal within subjects design
Assessing changes over time, months or years
Within subjects design
- participants own performance is basis of comparison
2. all participants exposed to all experimental conditions and receives all levels of the independent variable
Within subjects design can be used for
observational, correlational and experimental designs
Issue with within subject design
the experience with one condition can affect performance in subsequent conditions. control this by varying the order of presentation (counterbalancing)
Between groups design
participants randomly or not randomly selected for control and experimental, or conditioned and unconditioned groups
Equivalence of Groups (between groups design)
Same initial state for the experimental and the control groups before the manipulation of the independent variable
Random assignment (between groups design)
equally likely to be assigned to ether group, any differences in groups caused by chance
one way between groups experimental design with two levels of the independent variable
- Participants randomly assigned to contidions
a. 1. Initial state - equivalence among groups
a. 2 Manipulated independent variable
a. 3 measured dependent variable difference between groups
Correlations vs Experiements
Correlation identifies relationships between variables, experiments identifies causal relationships between two variables
Correlations
Relationships between two variables. a goes up when b goes up or a goes down when b goes up.
Cause and effect
A causes B or B causes A.
Problems with causality
Correlation cannot be used to determine causal relationships
Reciprocal causation problem
the two variables cause each other (positive attitudes and high self esteem)
Common causal variable problem
a variable that results in both the predictor and outcome variable (motivation predicts GPA and study hours)
Correlational Design
Study used to search for and define relationships among measured variables
Examples of predictor and outcome variables
strangers present and dogs barking students effort and grades train times and number of people food intake and obesity computer time and cancer rate age and rudeness
Types of sampling techniques
- Simple random sampling
- Systematic sampling
- Stratified sampling
- Cluster sampling
- Convenience sampling
- Snowball sampling
Types of probability sampling
- Simple random sampling
- Systematic sampling
- Stratified sampling
- Cluster sampling
Types of non-probability sampling
- Convenience sampling
2. Snowball sampling
Simple random sampling
all members of the population are equally likely to be chosen as part of the sample
Systematic sampling
Participants are not chosen randomly but instead are chosen according to some plan or strategy (like every third person)
Stratified sampling
random sampling intended to guarantee that the sample will be representative of the population on specific characteristics such as race or ethnicity or income level
Cluster sampling
for large populations groups of potential respondents are chosen instead of individual participants being selected. These groups are judged to be representative of the population and this can be a random selection.
Convenience sample
non probability where participants are chosen from a readily available situation (shoppers in a mall)
Snowball sample
used when researcher needs to contact members of a difficult to contact situation a researcher may ask potential participants to identify other possible participants (homeless)
Representative sample
approximately the same as the population in every important respect
Target population
the group to which the researcher wishes to generalize their findings
Single blind
either the participant or the experimenter do not know which group participants are in
Double blind
both the participants and experimenter do not know which group the participants are in
Threats to internal validity
- confounds
- instrumental effects
- subject mortality
- experimenter bias
- demand characteristics
Internal Validity
the extent to which the independent variable cause the dependent variable
External validity
the extent to which results can be generalized to the population as a whole
Reliability
consistency of the measurement procedure for the dependent variable
Trade off between validity and reliability
a valid measurement must be reliable, but a reliable measurement is not necessarily valid
Internal validity
are you studying what you claim to be studying or measuring what you say you are measuring
Internal validity requires
clear operational definitions of the dependent and independent variables
External validity
is concerned with the appropriateness of generalizing the results to an intended population
Reliability
are the results consistent?
Inter-rater reliability
More than one person collected the data and their data sets agreed
Replicability reliability
The same method in the same situation gets the same results so that the study can be repeated
Independent variable
the factor to be manipulated in an experiment
Dependent variable
the factor of the experiment that will be affected by a change in the independent variable
Confounding variables
extraneous variables that affect the measurement of the dependent variable
Operational definitions
descriptions of the the behaviors or aspects being studied that are in observable terms, apparent and measureable
Searching backwards
- relevant research will likely reference other relevant research
- identify recent relevant studies and look through their reverence sections
- not a substitute for a topic search but is a good supplementary strategy
Search by topic
- the most common search strategy
- abstract and keyword indexes structured for topic searches
- boolean operators OR AND NOT
- Identify all the work of authors you find who have done a lot of work in the field
Boolean operators
OR, AND & NOT
Literatures searches online library databases
- search by topic
- search by author, using authors who you know specialize in the area
- check out the references in relevant articles
- use citation indexes to find articles that cite classic articles in a field
Research Methods
Case studies Experiments Surveys Interviews Observation
Animal ethics
use of animals in research is very controversial
Animal rights groups
PETA, ALF
IACUC
acts as IRB for animal resarch
Animal investigator responsibilities
- design experiments with consideration for animal welfare
- administer research protocols that demonstrate consideration for animal welfare
- report results to the scientific community
Institutional Review Board (IRB)
- required by all institutions receiving federal funding
- consists of members from both scientific and unscientific disciplines
- at least one member must be a local community member who is not associated with the institution.
- research cannot be conducted without prior approval by the board
Institutional review board members
Must be from both scientific and non scientific disciplines and contain at least one non-institutional, local community member
Reasons for ethical research
- protect participants from physical and psychological harm
- provide freedom of choice of participation in research
- maintain awareness of power differentials
- Honestly describe the nature and use of the research
Scientific method steps
- define your research question
- research existing sources
- formulate a hypothesis
- design and conduce a study
- analyze data
- publish results
Define your research question (scientific method step 1)
observe behavior
study behavioral theories
Research existing sources (scientific method step 2)
Conduct literature searches.
Has anyone asked this question before?
What is known about your subject?
Formulate a hypothesis (scientific method step 3)
What do you think is the answer to your research question based on literature and your observations?
What is the null hypothesis?
Design and Conduct a Study (scientific method step 4)
- define your sample and sampling method
- define operational definitions, variables and identify confounds.
- Design an observational, experimental or quasi-experimental study
- Determine if your experiment is do-able with resources you have
- Decide your method of gathering and recording data
Methods of gathering and recording data
video survey interviews spreadsheets manuel
Analyze Data (scientific method step 5)
- What do you want to know form your data?
- What tests will best answer your research question?
- Interpret data and draw conclusions
- What do your results tell you in relationship to your hypothesis?
- How do your results compare to those in the literature you referenced?
Publish results (scientific method step 6)
Submit a report in APA style to peer reviewed journals
Scientific Method
- Ask a question
- Research existing sources
- Formulate a hypothesis
- Design and conduct a study
- Draw conclusions
- Report results
analysis of variance (ANOVA)
an inferential statistical test for comparing the means of three or more groups
correlation
a measure of the degree of relationship between two variables. The strength of the relationship is represented by the absolute value of the correlation coefficient. The direction of the relationship is represented by the sign of the correlation coefficient.
factorial design
a research design in which the effect of two ore more independent variables on a dependent variable is assessed
factors
independent variables
graphs of the cell means
an important technique for presenting the results of a factorial design so that they can be more easily interpreted
higher order designs
factorial designs that involve more than one independent variable
interaction effect
in a factorial design, the effect of a dependent measure on an independent variable with each level of each independent variable
main effect
the effect of an independent variable on a dependent measure within a factorial design
marginal means
the average of scores for each level of an independent variable, disregarding other independent variables. used in factorial designs to interpret main effects
reliability
the consistency with which the same results are obtained form the same test, instrument or procedure
quasi-experimental design
a type of research design in which non-equivalent groups are compared, a single group is observed a number of times, or both of these techniques are combined
pilot study
a smaller preliminary study conducted to answer questions about procedures for the full scale version of the investigation
regression toward the mean
the phenomenon that extreme scores tend to be less extreme upon retesting; they move toward the mean
Indicate the dependent and independent variables, number of levels for each independent variable and weather each variable is within-subjects of between-groups
Researchers observed male and female children (4, 5, and 6 years old) as they played with other children of the same age. For each child, the time spent in cooperative play with others was measured.
Dependent: cooperative play with others
Independent, gender 2 levels, age 3 levels
between-group variables
Indicate the dependent and independent variables, number of levels for each independent variable and weather each variable is within-subjects of between-groups
Individuals with anorexia and bulimia take a cognitive distortions test upon entering a treatment program for eating disorders and again three weeks later
Within-subjects and between groups design
Dependent: score on cognitive distortion tests
Independent: eating disorder 2 levels between-groups, testing two levels within subjects
Indicate the dependent and independent variables, number of levels for each independent variable and weather each variable is within-subjects of between-groups
A researcher is investigating the effect of noise level and subject matter on reading speed. The participants in this study were measured three times - once reading poetry, once reading a novel and reading a history text book. Half of the participants read in a noisy environment and half read in a quiet environment.
Within-subjects and between-groups design
Dependent: reading speed
Independent:
Noise level 2 levels noisy and quiet, between groups
subject matter 3 levels poetry, novel and history within groups
Indicate the dependent and independent variables, number of levels for each independent variable and weather each variable is within-subjects of between-groups
A researcher standing by the door of a grocery store stops half of the males and half of the females who enter alone and presents them with a free fight of a pen and small pad of paper. The other shoppers do not receive a gift. The researcher then notes how much money each shopper spends in the store
Between groups design
Dependent: how much money spent in store
Independent: gender two levels, free gift two levels
A researcher interested in the efefct of weateher on mood hypothesized that overcast days yield more depressed moods than sunny days. Participants were solicited from sections of a college course introduction to Economics. Participants in one group were asked, on a sunny day., to rate their mood on a scale from 1 depressed to 10 very happy. Participants in another group were asked, on an overcast day, to rate their mood. On both days the temperature was 68 degree; the test days were three weeks apart. If the research yielded higher mood scores on the good weather days, can the researcher claim that good weather caused a good mood? What are some alternative explanations for the results?
This is a correlational study and not an experiment, so researcher cannot claim weather caused mood change. The overcast day may have fallen closer to final exams, or day exams were returned to students. Alternatively something that occurred during those three weeks may have caused moods to fall
A researcher is interested in the relationship between stress and the severity of symptoms caused by the common cold a sample of volunteers is exposed to a cold virus and also indicated the amount of stress they have been feeling over the last week, using a scale of 1 to 10, 1 being very low stress and 19 being very high stress. Four days later the participants indicate on a 10-point scale the severity of any cold symptoms they are experiencing (with 1 being very mild or no symptoms and 10 being very severe symptoms)
If the researcher finds a correlation of .85 how might she describe the relationship between stress and cold symptoms?
The relationship would be strongly positively correlated
stress and the severity of symptoms caused by the common cold a sample of volunteers is exposed to a cold virus and also indicated the amount of stress they have been feeling over the last week, using a scale of 1 to 10, 1 being very low stress and 19 being very high stress. Four days later the participants indicate on a 10-point scale the severity of any cold symptoms they are experiencing (with 1 being very mild or no symptoms and 10 being very severe symptoms)
Indicate a correlation coefficient that would be interpreted as a weak negative relationship
A researcher finds a correlation of -.15.
stress and the severity of symptoms caused by the common cold a sample of volunteers is exposed to a cold virus and also indicated the amount of stress they have been feeling over the last week, using a scale of 1 to 10, 1 being very low stress and 19 being very high stress. Four days later the participants indicate on a 10-point scale the severity of any cold symptoms they are experiencing (with 1 being very mild or no symptoms and 10 being very severe symptoms)
what statistical technique would be sued to predict cold symptom severity?
Regression would be sued to predict cold symptom severity
multiple regression
a technique used to create a formula to gauge the relative importance of the various predictors
A researcher interested in the treatment of older people in our society asks your advice on observational methods. Together, you consider naturalistic observation, disguised participant observation and undisguised participant observation. Describe how each of these types of observational studies might be carried out.
A naturalistic observation study might involve unobtrusively watching how older people are treated in stores, on buses and in other public places. In a disguised participant study, you might dress as an oder person and observe how you are treated by others. In an undisguised participants study, you might join a group of senior citizens on an outing and observe how they are treated.
Do males ask and answer questions more often in lass than do females? A researcher wishes to observe student behavior in order to address this question. The researcher operationally defines “asking and answering questions” so that behaviors can be categorized easily. She suspects that males do speak more often in class, but is concerned that this bias might affect data collection. What advice would you give this person? What method of data collection would you suggest human observers or non human means? How could the researcher avoid observer bias (human or nonhuman)? Would the data collection involve a reactive measure? How could reactivity be minimized or avoided?
The researcher should probably ask a trained observer to make the observations, so that the researchers’ biases do not affect data collection. An alternative is to use a video camera, but the presence of the camera might affect the students’ behavior; after the data have been collected they will need to be scored by observers, who might still introduce biases. I would put trained observers in classrooms, preferably at the beginning of the semester hen the students will not know who is and who isn’t a student
Alternative forms reliability
an assessment of how well two forms of the same test yield comparable results
branching
the answer to a survey question determines which specific questions the person will see next
closed questions
survey, interview, or test questions that ask the respondent to chose from alternative potential answers
cluster samling
a technique in which clusters of elements that represent the population are identified and then all of the elements in those clusters are included in the sample
construct validity
the extent to which the concepts measured within the tool are actually being measured; it can be assessed by comparing the results of the new tool with the results of another established tool that measures the same construct
convenience or haphazard sampling
a sample composed of individuals who happened to be in the right place at the right time
criterion validity
measures how well the results of an instrument correlate with other outcomes or behaviors
Cronbach’s alpha
a statistical technique that compiles the correlatations of every item with every other item within a measurement tool
demographic questions
survey questions about the characteristics of a sample, such as average age, racial composition and socioeconomic status
double barreled questions
a survey interview or test question worded in such a manner as to ask more than one question at the same time.
elements
members of the same sample
face validity
the degree to which a measurement tool appears to be measuring what it is supposed to be measuring
filter question
a survey or interview question that instructs the respondent or interviewer as to what the next question should be for different answers by the respondent
funnel structure
survey or interview questions ordered from the most general question to the most specific question, and then sometimes back to more general
interviewer bias
the confound that arises when an interviewer’s behaviors, questions, or recording procedures result in data that are consistent with the interviewer’s personal beliefs and constitute an inaccurate record of the respondent’s true opinions or behavior.
leading questions
survey, interview or test questions in which information is presented in such a manner that the respondent is more likely to give the answer that the researcher wants
loaded questions
survey, interview or test questions that include non-neutral or emotionally laden terms
mail suverys
written, self-administered questionnaires
non probability sampling techniques
the sample i formed without considering the probability of each member of the population
open-ended questions
survey, interview, or test questions that do not provide specific options for answers but instead provide room or time for the respondent to go formulate his or her own response
personal interviews
a type of survey that involves a person-to-person meeting between the interviewer and respondent.
population
all of the individuals to whom a research project is meant to generalize
quota sampling
a sampling technique in which differing numbers of participants are chosen for each sample from various subgroups of a population by identifying convenient sources of subgroup members and soliciting participants from these sources
random sample
a sample in which the elements were selected randomly from a sampling frame
random selection
a list of all the members of a population; serves as the operational definition of the poulation
reliability
the consistency with which the same results are obtained from the same test, instruments or procedure
response rate
the extent to which people who receive a survey or are approached to complete an interview complete the survey or interview formula: (number of response) divided by (number in sample-ineligible and undeliverable requests) * 100
sample
a subset of the population
sampling bias
the extent to which a sample does not represent the underlying population
sampling frame
a list of a lll the members of a population; serves as the operation l definition of the population
snowball sampling
a sampling technique in which research participants are asked to identify other potential patricipants
socially desirable responses
responses that neglect what is deemed appropriate by society but do not necessarily reflect the respondent’s true beliefs, attitudes or behaviors
split-half reliability
the degree to which respondents replies to half of the items on a measurement tool is related to their replies on the other half of the items.
stratified random sampling
stratified sampling in which members of the sample are chosen randomly
stratified sampling
a sampling technique intended to guarantee that the sample will be representative of specific subgroups of the population, called strata. The sampling frame is divided in to such strata, and then the elements of the sample are chosen from the strata
telephone surveys
surveys conducted by phone
test-retest reliability
the degree to which a tool generates the same responses upon retesting
validity
the extent to which test, instrument or procedure is measuring what it purports to measure
confound
an uncontrolled, extraneous variable that yields alternative explanations for the results. limits internal validity
instrumentation effect
the confound arising hen a measuring device fails to measure in the same manner across observations
demand characteristic
cues inadvertently provided by the researcher, materials or setting that supply the pariticipant with information about the purpose of the investigation
experimenter bias
the confound arising when behavior differences in a study caused by the participation of different experimenters.
subject mortality or subject attrition
the loss of data when participants withdraw from a study or their data cannot be used
