Midterm 1 Flashcards
1
Q
Research variables
A
Independent: typically manipulated
Dependent: always measured
Control variable
2
Q
Experiment (3 steps)
A
Hypothesis
Experimental manipulation (independent variable)
Interesting outcome
3
Q
Categorical
A
Levels are categories
4
Q
Ordinal
A
Quantitative variable
Numerals represent a rank order, but distance is not equal
5
Q
Interval
A
Quantitative variable
Numerals have an equal distance, but no real zero
6
Q
Ratio
A
Quantitative variable
Numerals have an equal distance but there is a real zero
7
Q
Reliability
A
Consistency of results over time
Less spread: more reliable
How well we’ve measured something in a technical sense irrespective of what it is we’re measuring
8
Q
Internal reliability
A
Internally consistent
Correlate with itself
9
Q
Test-retest
A
Same results over time
Correlate with itself on 2 occasions
10
Q
Inter-rater
A
Two or more observers agree (scores correlate)
11
Q
Validity (definition)
A
We have properly captured the construct of interest
12
Q
Link between validity and reliability
A
For a measure to be valid, it has to be reliable
For a measure to be reliable, it doesn’t have to be valid
13
Q
Assessing validity (4 questions to ask yourself)
A
- Is it possible you’re measuring something else
- Are you measuring some part innacurently
- Are you measuring all important aspects
- Are you measuring it well enough to suit your intended argument
14
Q
Internal validity
A
Is there a better alternative explanantion?
15
Q
Construct validity
A
Capture the right thing
16
Q
Statistical validity
A
How big is the effect
17
Q
External validity
A
Representative sample
18
Q
Face validity
A
Does it seem to be valid
19
Q
Content validity
A
Capture all the aspects
20
Q
Criterion validity
A
Relate to a concrete outcome
21
Q
Convergent validity
A
Relate to other things it should
22
Q
Discriminant validity
A
Not relate to other things it shouldn’t
23
Q
Good design
A
Captured construct well
Good number of items
24
Q
Good answers
A
Order effects
Response sets
Faking
25
Order effect
Order of items in a questionnaire matter, so the order of questionnaires within a study, establishing a context for the participant
26
Acquiescence - Response set
Tendency to answer positively to all questions
| Threatens to construct validity
27
Fence-sitting - Response set
Selecting neutral answers
| Weakens construct validity
28
Social desirability - Faking
Trying to look better in someone's eyes
29
Malingering - Faking
People try to look bad
May try to say the survey is anonymous, but participants may not take it seriously
Include some reverse questions
30
Self-reports (4 reasons why they are not good)
Untrustworthy and useless because
1. The order of questions can affect the participants
2. The order of questionnaires can affect the participants
3. People don't answer thoughtfully
4. People try to emphasize that they are good or bad
31
Can we believe participants
Yes if we ask good questions
32
Correlations
- what is it
- the famous quote
The simplest form of association claim (bivariate correlation, zero-order correlation, linear regression)
A strong association doesn't imply causation
If there's is no correlation there's no causation
33
Features of correlations (4)
Only 2 variables
Both variables are measured
Min of -1 and max of +1
0 indicates no correlation
34
Understanding correlations (value of person's r)
```
Use Pearson's r
0,10= small
0,30= medium
0,50= strong
If it's smaller or bigger than expected, this might be a sign that there's a problem with our theory
```
35
If one variable is an IV and the other is DV, can correlation demonstrate causation?
Yes
| This is true even if your IV was only measured, not both measured and manipulated
36
Questioning validity (what's the assumption)
Assume that our sample is one homogeneous group, but they can vary relatively normally on both measures
37
Not homogeneous sample
May have some important problems
The sample has two or more subgroups that are different from each other
It can happen without any sign and detecting when it's a problem can be difficult
38
Internal validity of the correlation (sample size)
Outliers can influence internal validity, especially with small samples
Better to have large samples so we can increase our confidence
39
Rules of causation (3)
Covariance: cause (IV) must be related to the effect (DV)
Temporal precedence: cause before the effect
Internal validity: no other plausible explanation
40
Problems establishing covariance (think about the graph)
Correlations detect straight line association even when there is no association
41
Threats to experimental validity (3)
Design confounds, selection effects, order effects
42
Design confounds (variable)
Does another variable vary systematically with the IV (need for control variable)
43
Selection effects
Are different kinds of participants in the groups (independent-group design)
Error while selecting the participants to participate in a study
44
Order effects
Are later responses systematically affected by earlier ones (within-groups design)
45
Managing selection effects (2)
Random assignment
| Matching
46
Preventing order effects
For within-group design, we can encounter fatigue, practice effects, or carryover effects
Best solution is counterbalancing
47
Multiple groups
Posttest only
| Pretest and posttest
48
Single group
Concurrent measures
| Repeated measures
49
Independent-groups
Requires more people (control group)
| No contamination across IV levels
50
Within-groups
Requires fewer people
Participants are their own controls (power)
Order and demand effect
May not be possible
51
Confidence intervals
Confidence around the mean, correlation
If it includes 0, it's not statistically valid
Large sample for statistical validity
52
When IV doesn't have an effect
When manipulation is not effective at producing a change in the outcome, but it seems like it is
Manipulation seemed somewhat effective or not at all when it should have been
53
Type I error
False positive
| Reject the hypothesis when it's true
54
Type II error
False negative
| Fail to reject the hypothesis when it's false
55
Threats to internal validity
Can't always eliminate them all, need to recognize their impact
Threats can overlap each other
56
Testing effects
Special case of order effect
Testing process itself changes the outcome
Can happen in within-subject design
57
History effects
Unrelated event occurs during the study affecting the whole study and group
Can happen any time, impossible not to happen in longitudinal
58
Maturation effects
Natural or spontaneous change occurring
| Especially with kids
59
Comparisons groups
Best way to identify the presence of threats to internal validity
Eliminate if plausible alternatives occurred
60
Attrition
Outliers dropped out of the study, normalizing one group
Can't include the scores of people out dropped out of the study
Not applicable because we get rid of someone who didn't complete the whole study
61
Regression to the mean
Extreme scores at T1 are normal at T2
| Can happen if the mean shifts overtime
62
Instrument effects
Decay of equipment accuracy or coding method
| Can happen over a long period of time
63
Attrition vs regression to the mean
Pre rest looks about the same
1 person vs entire group
Group looks equivalent based on means but the number of dots doesn't (unusual score doesn't mean unusual mean)
Unusual score has a big effect on the mean, need to take out the unusual score of BOTH groups
Need to justify every time we take out a score
64
Observer bias
Ratings (or data) become adjusted to fit the hypothesis, researchers manipulate the data
Double-blind condition
Depends if we have observers
65
Demand effects
Participants attempt to produce the hypothesized outcome (participants change their answers to help the researchers)
Need to ask them at the end of the study their idea of the hypothesis
Between subject-design instead of within
Can happen if participants have more time to figure out the purpose of the study
66
Placebo effects
Participants can will themselves to improve, change
Control condition can tell us how big the placebo was
The downward slope doesn't guarantee a placebo effect
Depends if we are manipulating something or not
67
Probabilities
Not all threats are equally likely
Regression to the mean, testing, and demand effects are VERY likely
History, attrition, and instrumentation effects are pretty unlikely
Maturation and observer bias can be likely in the right contexts
Placebo is very likely in the right context
68
Recognizing threats (2 reasons why they help us)
Understanding the threats is very important
Help us to avoid accepting bad arguments as valid ones
Help us recognize ways to improve for follow-up studies
69
Third variable
Plausible alternative explanations can come in the form of alternative causal variables
Try to hold them constant as control variables
70
Multiple regression
- what does it help us with
- link with internal validity
- when is it used
- whats the letter
```
Helps us answer questions about an association while controlling some other association
If we take into account a potential confound, we improve internal validity
If we determine our preferred causal variable as a stronger influence than another cause, we improve internal validity
It is used when we want to predict the value of a variable based on the value of two or more other variables
Standardized beta (same principle as r)
R= combination of all factors
```
71
Recognizing the use of MR (3 expressions)
controlling for the effect of Z
taking into account the effect of Z
correcting for Z
72
Third variable problems
- role of MR
- allow us to do what
- how is the argument?
MR lets us control for other unwanted influences on the association of our IV and DV (Take the bivariate association and rule out any other influences we can think to measure )
Allows us to make more complex arguments
Argument is always causal when we use MR (always a predictor and outcome variable)
73
Other uses of MR
Not only good for ruling out third variable
Mediation
Moderation
74
Mediation
Occurs when some other variable helps us better understand the nature of the causal association between 2 variables
Third variable is present but it's not a problem
Mediators are involved in the causal association
75
Mediation vs third variable problem
Mediators are a part of the causal chain that connects an IV to a DV
A third variable is the only cause that makes an IV appear to be linked to a DV
76
Moderation
Occurs when some other variable controls whether your association does or doesn't appear
Moderators dictate what kind of association can exist between the IV and DV
77
Correlation and temporal precedence
Correlations are good at establishing covariance (MR will improve internal validity)
Correlations are not good at establishing temporal precedence
78
Longitudinal designs
- problem it addresses
- what does it do
- when is it useful
Help address the problem of temporal precedence with correlations
Follow participants over a long period of time
More work than cross-sectional design
Useful when experimental design is not possible
If 2 variables use correlations across variables
79
Cross-sectional correlations
Only calculate correlation at one-time point
| Used to examine if changes in one or more variable are related to changes in another variable(s)
80
Autocorrelations
Self-correlations, correlation with same variable
| Correlation with itself
81
Cross-lagged correlation
Across time and different variables
| Study in which two variables are measured once and then again at a later time
82
Parsimony
The degree of simplicity in a theory
Occam's Razor states that all else being equal, the solution that makes the fewest assumptions is usually the correct one
83
Pattern
Demonstrating that a series of different findings (preferably different studies) converge on the same solution
84
Pattern and parsimony
If we combine the two we really have something
If multiple studies show support for the same simple conclusion, it's much more likely that your proposed causal association is correct
