Sampling Flashcards
Which has higher INTERNAL validity: randomized or non-randomized designs
Randomized - participants are randomly assigned to levels of the IV
Which has higher EXTERNAL and CONSTRUCT validity: randomized or non-randomized designs?
Non-randomized
Variance BETWEEN participants
Different groups have different conditions and the groups are compared to each other (use random assignment to assign participants to groups)
Variance WITHIN participants
Each participant experiences different conditions - participants as own comparison
Selection (threat to internal validity)
Pre-existing differences between participants in different experimental conditions. Occurs when there is no random assignment
Maturation (threat to internal validity)
Participants/things measured change over time (boredom, decay, growth, development, etc.) (e.g. measuring concentration in children, but older children concentrate better)
History (threat to internal validity)
Interruption from an unwanted source affects DV (e.g. room getting warmer during a concentration test, which affects concentration)
Test effects (threat to internal validity)
The observation changes what it observes (e.g. due to repetition, subjects learn to do a concentration test faster (i.e., memory is measured instead of concentration))
Instrumentation (threat to internal validity)
The observation is not done consistently - changes in measurement procedures/devices (e.g. measuring concentration and using a new concentration test on day two of experiment)
Mortality (threat to internal validity)
Drop out (e.g. participants do not return for the post-(treatment) test)
Selection by maturation (threat to internal validity)
Different groups mature differently (e.g. examining groups of adolescents according to birth-assigned sex; testing effect of protein intake on development of strength)
Simple random (probability sampling)
Every element has an equal and independent chance of selection
Probability sampling
Every element of a population has a known chance of being included in the sample
Stratified random sampling (probability sampling)
Population is divided into distinct strata based on specific criteria, then samples are drawn from each stratum separately and combined
Cluster sampling (probability sampling)
Used for large population or geographically dispersed ones - create a list of clusters and select using simple/stratified random sampling
Systematic sampling
A random sample, with a fixed periodic interval, is selected from a larger population (e.g. every 5th, 10th, 15th element is selected)
Convenience sampling / Haphazard sampling (non-probability sampling)
- Select elements because they are at hand/easiest to access
- Prone to bias
Quota sampling (non-prob sampling)
- Selection of elements based on certain characteristics in the population (e.g. equal amounts of men and women)
- Bias through personal selections for convenience
Purposive sampling (non-prob sampling)
- Handpick cases believed to be representative on a population
- Compromises generalizability of findings
Snowball sampling (non-prob sampling)
- Multistage sampling technique - initially small and it expands as participants recruit others
- Start with a stratified random sample
Why use non-probability sampling?
- More economical
- Increased internal validity (but decreased external)
- Used for insight and ideas, not generalizability (useful in psychology)
Repeated measures design (randomized)
- exposes same participants participants to multiple treatments, allowing for variation to appear within the same individual
- randomly assigned to different conditions
- efficient and requires less participants (detects more individual fluctuations and biases)
Limitations and drawbacks of randomized experiments
- Experimental artifacts -> unintended effects on DV caused by experimental setting and not the IV
- Increased internal validity but decreased external -> findings are difficult to generalise
- “college sophomore” problem -> many experiments are conducted with college students so findings are not representative of the broader population
What is the aim of survey research?
To describe the distribution of attitudes, behaviours, and other characteristics within groups and examining their relationships
- NON EXPERIMENTAL (describes relationships, doesn’t establish causality)
Sources of error in survey research
- Coverage (some of the population is not in the sample - bias)
- Sampling (random differences between people)
- Measurement (bias from the way constructs are assessed)
- Non response (bias from low response rates - sample isn’t representative)
Survey designs (2 of them)
- Cross-sectional surveys –> data collected in a single point in time to estimate the prevalence of characteristics in a population
- Panel survey –> data collected from the same participant at multiple points in time to assess the stability or change (more costly but increases internal validity)
Modes of data collection in surveys
Questionnaires, face-to-face interviews, telephone interviews
Pros and cons of questionnaires
PROS:
- Cost effective
- Flexible for respondents
CONS:
- Interviewer bias and low response rates
- Anonymity of internet surveys leads to low accuracy
- Lack of control over answers
Pros and cons of face-to-face interviews
PROS:
- Ideal for open-ended responses and lengthy interviews
- High response rates
- Suitable for specific needs e.g. no internet/visual aids etc.
CONS:
- High cost and presence of interviewer = bias
- Difficult to access gated communities
Pros and cons of telephone interviews
PROS:
- Cost effective
- Efficient
- Reduced interviewer bias
CONS:
- Declining response rates
- Limited rapport with interviewee
- No visual aid
- Some questions may be too complex to do over the phone
Key ideas in non-experimental research
- No planned manipulation or intervention
- Sometimes can turn into quasi-experimental
- Useful when an event has already occurred
0 It can complement experimental research by exploring unplanned events - Opportunistic and flexible
- Includes surveys, questionnaires, observational, and archival research
Main idea surrounding naturalness in research
observational and archival methods to offer unobtrusive ways to study behaviour (participants are unaware of certain parts)
What are the 3 dimensions of naturalness in research?
- Behaviour: occurs independently of researchers influence
- Setting: Context not created for research purposes i.e. public spaces –> makes participants act more naturally
- Event: Incidents that happen without research intervention
What are collateral reports (indirect method of measurement)
Someone other than the participant provides information about the participant
- detects biases in self-reports
- Sometimes third-party reports disagree and require more research
- Increases the cost of study and has longer recruitment
What is observation (indirect method of measurement)
- Does not rely on participants = unbiased judges
- Take note of subtle, nonverbal cues
- Requires unobtrusive observation
What are psychological measures (as an indirect method of measurement)
- interplay of physiological systems (endocrine/cardiovascular)
- despite knowing, the participants cannot control the outcome
- Real time measurement
- Overwhelming information, numerous control variables, lots of expertise needed
- Expensive and sensitive equipment
Criteria to create good questions for surveys
- TERMINOLOGY –> avoid jargon and ambiguity
- AVOID DOUBLE DENIAL –> no denial at all is best
- KISS –> Keep It Short and Simple but specific
- Avoid multi-dimensional questions
- Use OBJECTIVE and non-suggestive language
