Exam 3 Flashcards
the most common research data collection methods in 205?
questionnaires and surveys
what are examples of large scale survey industries?
Pew- political
Gallup- consumer opinion
youGov- international public opinion
nielsenn- tv ratings
key parts of a questionairre
items
variables
responses
each question within a questionnaire is called an
item
each item in a questionnaire can be used as a
variable
(can be a part or all of an operational definition)
responses from a questionnaire should be treated as
confidential
what is a criterion variable?
the survey form of the dependent variable
the focus of your survey
what is an example of a criterion variable?
Did you watch the superbowl?
what is a predictor variable?
the survey form of an IV
the factors that influence he criterion
what is a demographic variable?
variable such as age gender or access to TV
(can also be a predictor variable)
what is an example of a demographic variable question?
what demographic is most likely to watch the superbowl?
data depends on
how the question is answered
what are the 4 questions types?
- open ended
- restricted items
- partially open ended
- rating scale
what should you do before administering your questions of surveys to participants?
run pilot testing
what are open ended items>
allow participants to respond using their own numbers words or lists
comes in 3 types:
numerical open ended
descriptive open ended
list open ended
numerical open ended questions example
how many times have you watched the jersey shore?
descriptive open ended example
what did you like about the jersey shore?
list open ended question example
list shows like the jersey shore
what are restricted items?
they provide specific alternatives,
the questions need to be listed in a logical order
a whole spectrum of answers need to be provided
what is an example of an restricted item
Did the queens gambit change your interest in chess?
a.) decreased my interest
b.) increased my interest
c.) did not impact my interest
what are partially open ended items?
provides specific response alternates plus an other category where participants can provide clarification as needed
what is an example of a partially open ended question?
when do you usually stream movies?
a.)morning
b.) midday
c.) evening/night
____ other
what is a rating scale item?
provides a graded response to a question
-they include points and anchors
what is an example of a rating scale question?
would you buy Disney+ for a single movie?
very 1 2 3 4 5 very
unlikely likely
what is a point?
the value of a choice (usually on a scale of 1-5/7)
what is an anchor?
labels that give meaning to the points
ex.) very unlikely
what are the specific rating scale names?
visual analog scale
likert scale
likert type scale
what is a visual analog scale?
removes all points
participants mark a line where they think is appropriate
line placement is measured to retrieve a score
what are likert scales?
the extent to which a participant agrees about a statement using a 5 point scale
the anchors always range from strongly agree to strongly disagree
what is a likert type scale?
similar to a likert scale but doesnt use agree
what is an example of a likert type scale
angela duckworths grit scale
it is important that your survey has
- continuity and logic
aka related questions should be ordered together - not leading the participant
aka the questions should not give away your hypthesis - understandable and easy to read
- not too long
what are the methods of distribution for surveys?
- phone
- internet
- face to face
mail surveys
- mail the questionnaire to the participant
+it is the best way to get a near random sample because you randomly select from a public list
-has a very high nonresponse bias because most people do not respond
how do you encourage people to respond to mail surveys?
include money in the envelope
make multiple attempts
telephone surveys
call the participants and ask them the questions
- either have real people calling or have an automated response system
+can access older adults better
-high non response bias
(land lines used to make it near random sampling method)
internet survey
create the questionnaire online and either send a direct file, link, or social media post to reach the participant
+can access very large samples
- biased toward tech users and younger generatiosn
what are examples of internet survey tools
qualtrics
google forms
microsoft forms
face to face interviews
talk to each participant directly
+highest response rate
-less likely to admit unpopular behavior and opinions
- more susceptible to interviewer bias due to the tone or demeanor of the questions
why do face to face interviewers have the highest response rate?
there is more social pressure to participate
response rate is highest when it is with members of the opposite sex
what are the three types of face to face interviews>
structured, unstructured, and semistructured interviews
what is a structured interview?
ask specific questions by reading prepared questions to the participant
what is an undtructured interview?
no predetermined questions
-more similar to a discussion or conversation
can yield unexpected information
what is a semistructured interview?
elements of both other types
what is a mixed model method?
use multiple data collection methods to collect data
+diversifies your example and increases the nukmber of responses
-new opportunities for errors
(responses will vary depending on how the survey is received)
what is survey representativeness?
the survey sample should represent the intended population
what are some ways to achieve a representative sample?
- correct gender ratio
-collect a nonbiased sample
(note: even random sample can be biased by chance
what is stratified sampling?
determines the relevant strata in your population and then select your sample to represent each strata equally
what is a strata?
groups within your intended variable
what is an example of a strata? How would you use that to create a stratified sample?
class sizes,, you would then take 10 students from each of the class size groups
what is a proportionate sample?
you select your sample to select the strata proportionately
what is an example of how you would use this in a proportionate sample?
you would take 15% from class size A and then 45 from class size B,, etc.
how many people should be used in a sample?
enough to avoid sampling error
(get as many as possible)
what is sampling error?
when you use a sample too small to represent you population
what happens after you collect your data?
- organizing the data
-summarizing the data - graphing data
- describing data
we may use a ____ approach after collecting data
statistical
what types of statistical approaches can we use?
descriptive statistics
-hypothesis testing
when checking for data errors check for…
- missing data
-someone didnt fill the question in - impossible values
-someones age is writted as 212 - ambiguous resopnses
- youre unsure what their answer is
what do we use spreadsheets for (excel)
a lot of things but namely organizing data
what is included in your data summary sheet?
columns and rows
columns should include
-1 variable per
(ie demographics, age, question, etc.)
a row should include
- 1 participant per (they get assigned a number)
why do we use numbers for participants?
anonymity and organization purposes
in the data summary sheet you should avoid
using text for data values because this limits your ability to analyze the data later
what is a dummy variable
used to represent categories
when you assign a number value to a response so that you can analyze the data later
graphs….
visually represent the data
graphs have
2 dimensions (x)(y)
the horizontal axis is the
x axis
predictor variable
levels of your independent variable
the vertical axis is the
y axis
criterion variable
the performance of the dependent variable
what is nominal data?
nata that fits into distinct categories
(restricted items)
what is continuous data?
- numerical open ended questions and rating scale items
what type of graph goes with bar graphs?
nominal data
what type of graph goes with line graphs?
continuous data (time)
what type of graph goes with a scatter plot?
continuous data,, anything but time
every graph is designed
the same way
how do you describe direction and strength of a trend on a scatterplot?
positive relationship
-both increase or decrease
negative relationship
- one goes up while the other goes down
strong relationship
-variable change together
weak/no relationship
-variables change independently of one another
what are common patterns in data?
monotonic and nonmonotonic
what is a monotonic pattern?
data changes uniformly (either up or down)
what is a nonmonotonic pattern?
data increases and decreases
- there is performance “sweet spot”
what are the qualities that create a scientific graph?
-x and y axis labels
- sensible axis scale
- nothing in the background
-border around the chart space
- no numbers in the chart space
- font large enough to read
- high contrast
- no 3 d
- no title
- figure caption below graph
numbers and graphs are often also used to
sway an audience
how do numbers and graphs sway audiences?
incorrect numbers
correct numbers misinterpreted by a nonscientist
numbers are selectively used to be misleading
graphs are designed to mislead
what is big duck data?
when design relies on graphic themes instead of accurate information
(more commonly used in business)
where is a good place to start when trying to understand your data?
frequency
-rating scale questions
- continuous demographic data
what should you use when trying to understand your data?
frequency
data distribution
outliers
measures of central tendency
what are questions you can ask to better understand your data distribution
do your measures have a floor or ceiling effect?
is your sample biased
distribution describes…
your sample
what does normal distribution look like?
the sample data represents the population
humps in the middle of the graph
does a skewed distribution look like?
the sample tends to one end or the other
humps toward the beginning or end
what is an outlier?
extreme scores that pull on your average
what are the measures of central tendency?
mean
median
mode
what does the mean measure
thw average of all the data points
what does the mode meaure
the most common value
what does the median measure?
the middle value
how do you test the effect of an outlier on the measures of central tendency?
put it into excel
what in excel is used for calculations
“functions”
how do you activate a function in excel?
type “=” into a cell
what is the function of a histogram?
used to show outliers
what is the measure of distribution spread?
standard deviation
what is standard deviation?
the average distance of individual values from the mean
what function do you use in excel to get the standard deviation
STDEV.S
whenever you report a mean you also need to
report a standard deviation
how do we use he median?
median split
what is a median split
create two nominal predictor variables by splitting a continuous variable
-get a high group (above median) and a low group (below median)
in median split you use ____ to quantify your low and high groups
dummy variables
what are the steps in creating a median split:
- list all predictor values in order
- find median for predictor
- create new predictor categories
- compare criterion based categories
what is significance?
determining if two groups are “actually” different
(p-value)
calculated using a T-test
how do you calculate a t-test in excel?
- select data from both groups
- select two tail (alpha level)
- select equal variance
why do we use two tail tests?
they have better accuracy
what does the p value need to be in order to be signficant?
<.05 or 5%
what do you use to evaluate variable relationships
pearsons correlation coefficient (r-value)
what does correlation coefficient (r-value) do?
gives strength and direction of relationship
(each point on a scatterplot = 1 participant)
how to calculate correlation coefficient in excel
=Correll( two arrays of numbers)
the direction of a relationship can either be
positive or negative
what does it mean to have a strong relationship?
.7<
what value is considered very weak correlation
.1-.29
what value is considered weak correlation
.3-.49
what value is considered moderate correlation?
.5-.69
correlation requires
variability
(low variability=low variation)
(this applies to skewed data too)
correlation assumes
a linear realtionship
data in a nonlinear trend that does not appear correlated but can be is called
nonmonotonic
what was the 2014 Kramer study?
collaboration between facebook and cornell
-studied social emotional contagion
- almost 700k users recieved an altered news feed with either fewer positive posts or fewer negative posts
-then they monitored that groups future posting habits
what did the Kramer study find
our media feed impacts our mood
the kramer study recieved
negative criticism towards the researchers and the journal that published the study
how did he kramer study get published even though it was unethical
the journal relied on the Cornells IRB, and the IRB was like idc your doing it on facebook and I want to have our name on a publication
what journal published the Kramer study?
Proceedings of the national academy of sciences
those who teach, research, or practice psychological science should:
and
should not:
promote accuracy, honesty, and truthfulness
steal cheat, engage in fraud, misrepresent facts
failure to practice science ethically….
erodes trust
what is a prime example of a study that misused science?
wakefield (1998) vaccines and autism study
what was the Wakefield study about?
wakefield conducted a study that linked vaccines to the development of autism
(has 3000 citations)
gave rise to the modern anti vaccine movement
lead to several measles outbreaks
why did the wakefield study misuse science so badly?
they lied about a lot of their data
- they left out some patient data
- they only used data from about 12 participants
- they made up and altered medical records
- funded by a law firm suing a MMR drug co.
-relied on parents own beliefs and memories of symptoms and timelines in order to diagnose autism
- the redaction took 12 years
(all 9 of his coauthors had no idea what Wakefield was doing)
(redaction means when the journal finally came out and said that they no longer support the study
what can potentially harm research integrity?
- plagiarism
- falsification
- fabricating
what is plagiarism?
claiming another’s ideas, processes, or results without giving credit
what is falsification?
altering or omitting data or parts of the method without acknowledgement
what is fabrication?
reporting data or results that were made up
what percentage of research involves misconduct?
25%
what percentage of researchers see misconduct at some point in their careers?
50%
why do people think its okay to commit misconduct?
“its okay if its just me”
self esteem as a scientist
(people want to know that they can do good science and feel good about their work, so they lie)
what are some other reasons researchers may commit misconduct?
conflicts of interest
- want finnancial support for future research
- want academic promotion and tenure
(publish or perish)
how can we prevent reaearch misconduct?
careful peer review
encourage replication
create a healthy research culture
why is careful peer review important to stopping misconduct?
by making all data available to reviewers it will make it easier to pick out mistakes or lies
why is study replication important to stopping misconduct?
it weakens incorrect results
by publishing nonsignificant findings it helps weaken incorrect siginificant findings
why is it important to create a healthy research culture when preventing misconduct?
it encourages whistleblowing
penalties can be used as deterrents
why are people so discouraged from whistleblowing?
70% of whistleblowers received negative consequences
-pressure to drop charges
- ostracization
-fired or not promoted
- counter complaints
