Trying not to fail Flashcards
What type of RQs are there?
Exploratory
Explanatory
Descriptive
What are exploratory RQs for, and what type of RD is often used for them?
Pave way for further questions, understand what happens (often flexible RD)
What are descriptive RQs for, and what type of research are they often used in?
Portray accurate profile of people or situations; comparison
NOT: testing relationships or predicting outcomes
Quantitative study/ after qualitative study
What are explanatory RQs for, and what type of RD is often used for them?
Explain situation, problem, pattern; find relationships between phenomena
Looking for causality
What type of study fits with a fixed research design?
Experiments, quasi-experiments, non-experimental studies
What type of study fits with a flexible research design?
Case studies, ethnographic research, grounded theory
What RD and type of study are usually used for descriptive RQs?
Fixed, non-experimental
What RD and type of study are usually used for explanatory RQs?
Fixed, (non)experimental
What RD and type of study are usually used for exploratory RQs?
Flexible, whatever
What are limitations of experimental designs?
IV manipulation not always possible/ ethical
Random assignment ≠ equivalent groups (tho the more the better)
Sometimes other designs work better
What are internal validity threats? (9)
Maturation
History
Test effects
Instrumentation
Regression to the mean
Selection
Mortality & attrition
Interactions with selection (e.g., history)
Uncertainty over causal influences (A -> B vs. B -> A?)
What are statistical validity threats? (5)
Low power
Fishing
Unreliable measurement instruments
Unstandardised procedure
Coincidental differences between groups/ in experimental situation
What are construct validity threats? (7)
Construct underrepresentation
Surplus construct irrelevancies
Mono method bias
Demoralised control group
Fear of evaluation
Research expectations
Hypothesis guessing
What are external validity threats?
Specificity to:
group
context
historical influences
constructs within group
What are the four types of validity mentioned in the course?
Statistical + internal + external + construct
What types of IV manipulations are there?
Stimulus (e.g., training vs. no training)
Instructional (e.g., cooperate vs. compete)
Contextual (e.g., cluttered vs. clean room)
In what ways can the DV be measured?
Subjective (report)
Cognitive (do task)
Behavioural
Neurobiological (fMRI)
Physiological (heart rate)
What are the three necessary conditions to establish a causal relationship?
- Relationship condition (A and B related)
- Temporal antecedence condition (A prior to B)
- Lack of alternative explanations condition (no alternative explanation C)
How can you increase external validity?
Random sampling
Deliberate sampling for heterogeneity or maximum differences
Generalising to modal instance (purposely sampling for particular instance like age)
Replication (to check external validity)
What are ways to measure variables?
Tests
Questionnaires
Observations
Interviews
Objective data (e.g., salary)
What is common method variance?
Variance attributable to measurement method > construct
What are problems with observation as a measurement method?
Reactivity of participants to being observed
Observational biases: selective attention, selective encoding, memory, interpersonal factors
What are problems with questionnaires as a measurement method?
Subjective
Are participants honest/ serious? Do understand questions?
Common method variance
Non-response bias
What are the seven steps to choosing a questionnaire?
- Theoretical framework
- Concept analysis/ specification
- General item attributes (abstract -> behaviour, cognition; determine sub scales)
- Generate items
- Evaluate items
- Create questionnaire
- Evaluate questionnaire
What is important when generating questionnaire items?
Simple language (clarity, brevity)
Clear timeframe (bounded recall, restricted recall)
What should be avoided when generating questionnaire items?
Open-ended questions
Negative questions
Frequency references
Leading questions
Double-barred questions
What is the Socrates effect?
Filling in one item influences how you fill in later items
What are shared biases by participants for the same method?
Social desirability
Negative affect
Acquiescence
Illusory correlations
Halo bias (e.g., rating someone similarly for different performance aspects)
How can you see common method variance?
Multi-trait multi-method matrix
What types of validity can you see in a MTMM matrix?
Convergent validity (similar constructs should be related)
Discriminant validity (dissimilar constructs should be unrelated)
How can common method variance be reduced?
Using different sources to measure IV and DV
Separate measurement of IV and DV (temporally, proximally, psychologically)
Statistically control for bias
What are the 6 research steps?
- RQ
- Measurement method
- Sampling strategy
- Measurement strategy
[data collection] - Analyse data
- Evaluate results
How can you deal with extraneous variables in experiments?
Control by randomisation
Control by matching or holding constant
Sampling
When does an extraneous variable become a confound?
When it systematically varies both the IV and the DV
What kind of true experimental designs are there?
Matched design
Repeated-measures design
Two-group designs
- Post-test only RCT
- Post-test only two treatment comparison
- Pre-test post-test two treatment comparison
- Pre-test post-test RCT
3+ group simple designs
Factorial design
Parametric design
What kind of quasi experimental designs are there?
Pre-experimental
- Single-group post-test only (-)
- Post-test only non-equivalent groups (-)
- Pre-test post-test single group design (-)
- Pre-test post-test non-equivalent group
Interrupted time series (repeated measures pre and post intervention)
Regression-discontinuity (below/ above set criterion value into different groups)
What are types of single-case experiments?
[A = baseline phase; B = intervention phase]
A-B designs (similar to interrupted time series)
A-B-A designs
A-B-A-B designs
Multiple baseline designs
- Across settings
- Across behaviours
- Across participants
Why do non-experimental fixed designs exist?
Some things cannot be modified (e.g., age), should not be modified (e.g., smoking), are not feasible to modify (e.g., placement in school)
What are types of non-experimental fixed designs?
Descriptive
Explanatory
Predictive
What are the three types of time-dimensions?
Cross-sectional (1 point)
Longitudinal (2+ points)
- Panel designs (same participants)
- Trend designs (different participants)
Retrospective (simulate longitudinal)
What are three common design problems that lead to rejection from AMJ?
Mismatch between research question and design (e.g., cross-sectional design for causal RQ, wrong sampling)
Measurement and operationalisation (wrong adaptation of pre-existing measures, common method variance)
Incomplete model (proper use of control variables, mediators operationalised)
What are the three conditions control variables should meet for being included in a study?
- Strong expectation that it’s correlated with DV
- Strong expectation that it’s related with IV
- Logical reason why it’s not a more central variable
What do Spencer et al. thinking about measurement-of-mediation designs?
Nah g
When should one use a measurement-of-mediation design?
Mediator hard to manipulate but easy to measure
When should one use a moderation-of-process design?
Mediator easy to manipulate but hard to measure
When should one use a experimental-causal-chain design?
Mediator easy to manipulate and easy to measure
What is a census?
When u survey a whole entire population
What types of probability sampling are there? (5)
Simple random sampling (e.g., lottery)
Systematic sampling (e.g., every 5th name)
Stratified random sampling (e.g., from subgroups) - can be (dis)proportionate
Cluster sampling (when population dispersed and large)
Multi-stage sampling (cluster sampling, can be stratified; e.g., school - class - child sample)
What kind of non-probability samples are there? (10)
Quota sampling (from various population elements)
Dimensional sampling (kinda quota sample; include important survey dimension in sampling)
Convenience sampling (near & convenient participants)
Purposive sampling (specific sample; used in grounded theory)
Snowball sampling (participants find new ones)
Time sampling (across time; use specific space at specific time)
Heterogeneous sampling (wide varying on value)
Homogeneous sampling (little varying on value)
Extreme case sampling (extreme values)
Rare element sampling (low frequency in population)
What is a sociometric scale?
Describes relationship between individuals in a group
How can a time frame be provided in questionnaire questions? (5)
Bounded recall (since last…)
Restricted recall period (in last 3 months)
Average (how many on average)
Marks (since beginning of school year)
Cues (when u won…)
What are the three main observational method types?
Participant observation (flexible, qualitative)
Structured observation (fixed, quantitative)
Unobtrusive observation (non-participatory role; can be (un)structured)
What are four advantages of observation?
Directness
Triangulation (help complement other info)
No social desirability effect
Lack of artificiality -> ecological validity
What are three disadvantages of observation?
Reactivity (if observer affects situation)
Logical problem (behaviour measured or not?)
Time consuming (less if structured)
What’s the dichotomy of observation as a method?
Formal observations (structured, direction; pre-specified elements) -> reliability, validity
Informal observations (more freedom) -> complete, complex data
For which situations is participant observation suited?
Small groups
Short events
Frequent events
Accessible events
Enough time
U wanna find out what’s going on
What does participant observation entail?
Becoming member of observed group to share “symbolic” world, habits, norms, etc.
What are participant observation roles?
Complete participant (observer doesn’t share observer status)
Participant-as-observer (participating & observing)
Marginal participant (less participation; e.g., bus passenger)
Observer-as-participant (no participating but consensual observing)
What are the five main steps of analytic induction?
- Rough definition
- Initial hypothesis
- Study situation in context of hypotheses
- Edit hypotheses or definition
- Repeat
What observational biases come with participant observation?
Selective attention (even out attention)
Selective encoding (subconscious)
Selective memory (write shit up)
Interpersonal factors (biases)
What are two truths regarding mono method bias Spector names?
Measurement affects data
Same method and same bias -> correlation inflated
What are five sources thought to be central to mono method bias?
Social desirability
Negative affectivity
Acquiescence
Illusory correlations
Halo bias
How can u deal with mono method bias?
Rethink - is it actually there?
Different sources to measure IV and DV
Separate IV and DV measurement (temporally, psychologically w cover story, proximally in different conditions)
What two viewpoints are there about real-world research?
Evaluation viewpoint (understand, communicate)
Change viewpoint (suggest how to change shit)
What are the three types of gap spotting?
Confusion spotting (competing explanations)
Neglect spotting (under-researched)
Application spotting (extending existing literature)
What are the three key ideas of abduction?
- Explaining patterns of data, build hypothesis
- Entertaining multiple hypotheses (comparative theory evaluation)
- Inference to best explanation (similar to grounded theory) - best explanation given evidence
What are criteria for theory evaluation? (5)
Fruitfulness, parsimony, scope, progressiveness, consistency
What are three famous flexible design research strategies?
Case study
Ethnographic study
Grounded theory study
What components of a study should be decided on before and after RQ?
Before: Purpose, conceptual framework
After: methods, sampling
What’s the ecological fallacy?
Making inferences about individuals from aggregate data
Validity =
accuracy
Reliability =
consistency
What are the two ways in which lab experiments may lack realism?
Experimental realism - situation not: realistic, involves & impacts participants
Mundane realism - situation not realistic in real world
What are two general types of bias specific to experiments?
Demand characteristics (do what u think u should)
Expectancy effects (experimenters see what they wanna)
