PSYC*2360 Chapter 4: Tools for Building Better Designs Flashcards
Which research design involves manipulating all independent variables and randomly assigning participants to groups?
True experiments
What is a quasi-experimental design?
A research design in which the researcher can’t manipulate the independent variable or use random assignment
Are non-equivalent, posttest only designs and non-equivalent, pretest-posttest designs experimental or quasi-experimental?
Quasi experimental
What is the difference between non-equivalent group, posttest only and non-equivalent group, pretest-posttest designs?
- Posttest only: Outcome measures are administered to two or more groups without random assignment
- Pretest-posttest: Baseline measures of groups are taken before an intervention, then an outcome measure is administered
What are quasi-independent variables?
Variables treated as if they are independent variables in an experimental design, even though they can’t be manipulated
What is self-report?
Any measurement technique that directly asks participants how they think or feel
What are the two most common examples of self-report measures?
- Interviews
- Surveys/questionnaires
What are the two main advantages of self-report measures?
- Relatively cheap and easy to administer
- Receive first-hand information from the source
Social desirability, demand characteristics, and retrospective bias are the main drawbacks of which type of measure?
Self-report
What is social desirability?
The tendency for respondents to give answers that make them look good
What are demand characteristics?
Cues that potentially make the participants aware of what the researcher expects
What is retrospective bias?
The tendency to view or interpret past events inaccurately
What must researchers do to properly measure behaviour?
They must find a way to quantify it
What are the three main types of behavioural measures?
- Behavioural trace
- Behavioural observation
- Behavioural choice
What is a behavioural trace?
A behavioural measure that relies on evidence left behind by a participant who is longer present to determine if a behaviour occurred
What are the two main advantages of behavioural trace measures?
- Doesn’t require the participants’ cooperation
- Avoids issues like social desirability and demand characteristics
What is the main disadvantage of behavioural trace measures?
The interpretation of a behavioural trace’s meaning depends on the researcher to draw inferences
What do behavioural observations rely on?
Directly seeing or observing behaviour as it occurs
T or F: A drawback of behavioural observation is that it cannot go beyond what can be visually observed.
False
Which behavioural measure involves participants making a purposeful selection from several options?
Behavioural choices
What are the two main disadvantages of behavioural measures?
- They are often time-consuming, expensive, and cumbersome
- Introduce the possibility for participant reactivity and demand characteristics
What is participant reactivity?
When participants act differently or unnaturally because they know someone is watching them
T or F: Behavioural measures tend to provide findings that are more relatable to the general public.
True
What are unobtrusive measures?
Strategies that allow for observation and assessment without a participant’s awareness
What is the difference between a raw scare and a true score?
- Raw score: The actual score, comprised of a true score and error
- True score: What the raw score would be if the test were a perfect measure and was uninfluenced by extraneous factors
What are the extraneous influences that cause a raw score to deviate from the true score referred to as?
Errors
A variation from the measure’s true score due to unsystematic or chance factors that occur in a haphazard, disorganized, or arbitrary way is known as what?
Random error
T or F: It is possible to remove random error from a study.
False
T or F: Bias is also known as a systematic error.
True
What is bias?
An error that systematically pushes scores in one direction
T or F: Statistics can be used to address bias.
False
Which form of error is more problematic, random error or bias?
Bias
What is standardization?
Keeping experimental conditions the same for everyone and as free from variation as possible
T or F: Standardization helps minimize bias, but can’t remove random error.
True
What is observer/scorer bias?
The misinterpretation of an observation based on existing beliefs, previous experiences, or expectations
What does the sensitivity of a measurement refer to?
The range of data a researcher can gather from a particular instrument
To strike the right balance between being overly sensitive and not sensitive enough, it is recommended to use how many options in self-report measures?
5-9
What is the ceiling effect?
When all responses of a measure fall at the top end of the scale
In terms of the boundaries of a measurement tool, what might cause a ceiling effect?
The upper boundary being set too low
If the independent variable is unable to affect the dependent variable, which is more likely, a ceiling effect or a floor effect?
A ceiling effect
What is the floor effect?
When all responses of a measure fall at the bottom end of the scale
In terms of the boundaries of a measurement tool, what might cause a floor effect?
The lower boundary being set too high
What is the difference between reliability and validity?
- Reliability: The extent to which a measure will produce the same results each time it is used
- Validity: The extent to which a research method measures what it is intended to measure
Does random error threaten reliability or validity?
Reliability
Does bias threaten reliability or validity?
Validity
T or F: Without validity, there is no reliability.
False. Without reliability, there is no validity.
T or F: Even with reliability, a measure can either be valid or invalid.
True
What is the entire group of interest in a study referred to as?
The population
What is the subset of a population from which the researcher collects data referred to as?
A sample
What is the explicit strategy used for recruiting participants from the population referred to as?
The sampling plan
What is a representative sample?
A sample with specific features that characterize the population of interest
What is probability sampling?
A sampling approach in which everyone from a given population has an equal chance of being selected to participate in a study
What are three types of probability sampling?
- Simple random sampling
- Stratified random sampling
- Cluster random sampling
What is the procedure for simple random sampling?
A subset of individuals is randomly selected from the entire population
T or F: Access to the entire population is not necessary for simple random sampling.
False
What is the procedure for stratified random sampling?
The population is divided into strata (subpopulations), then simple random sampling is used to select participants from each stratum in proportion to the population at large
What is the procedure for cluster random sampling?
The population is divided into groups/clusters, then simple random sampling is used to select which clusters participate
T or F: In cluster random sampling, clusters are normally naturally occurring.
True
What is non-probability sampling?
A sampling approach in which everyone from a given population does not have an equal chance of being selected to participate in a study
What are four types of non-probability sampling?
- Convenience sampling
- Quota sampling
- Purposive sampling
- Snowball sampling
What is the procedure for convenience sampling?
Involves nonrandom selections of participants who are readily available to the researcher
What is the procedure for quota sampling?
Researchers choose ahead of time that they want their sample to hold certain characteristics, then freely choose any participant that helps meet the quota
What is the procedure for purposive sampling?
The researcher chooses a sample based on whom they think would be appropriate or qualified for the study
Which type of non-probability sampling is most often used when a limited number of people have expertise in the area under investigation?
Purposive sampling
What is the procedure for snowball sampling?
Existing study participants recruit future participants from among their acquaintances
How does snowball sampling introduce bias into a sample?
The more social connections a person has, the higher their chance of selection
What does non-response bias refer to?
The chance that those who refused to participate may systematically differ in a meaningful way from those who did
Comparing the results of individuals who refused to participate at first, then agreed, with individuals who initially accepted, may be helpful when making inferences about which type of bias?
Non-response bias
What is the volunteer subject problem?
The concern about the generalizability of conclusions from volunteer participants because they may differ from non-volunteers in meaningful ways
What is the college sophomore problem?
The concern of generalizability, given that researchers often use convenience samples consisting of college/university students