week 8: research design Flashcards
research design is a plan that includes (4 things)
- participant selection criteria
- controlling extraneous variables
- variables of interest and conducting observations
- ensuring ethical procedure
is a good design enough to show the scientific value of a research project?
no! a good hypothesis is as important
what are 3 things that make a good hypothesis
- propose a clear relationship
- testable
- must “make sense” in other words it can’t just be a random guess
what is a good research design created to do
- answer the research question/questions
* control extraneous variables to ensure a high internal validity level
4 steps to building a good research study
- identify the population
- sampling protocol
- select appropriate design
- select appropriate statistical analysis
population
refers to all of the persons of interest for a particular study
- defined in the planning phase of the study
- all members must have one or more predetermined characteristic in common
- –note that population can also be defined as a group unit (all first grade classrooms in the U.S.)
parameters
numbers from observations on the entire population
statistics
numbers from observations on a sample
intended population
all persons whom the researchers want to apply their results to
accessible population
the group from which the researchers actually recruit their participants
census
when researchers attempt to gather data from all members of a population
why are participant selection criteria critical
- first they need to be based on established standards
- need a representative sample
- affect internal validity, ability to generalize, and ability to replicate the study
what is a representative sample
- includes individuals from each constituency in the target population including minorities
- simple sampling is usually sufficient, but if not then uses stratified sampling
what is a sample
refers to the persons within the population that actually participate in the study
- –use term participants not subjects
- –important to recruit a sample that represents the population well
- –avoid systematic exclusion
inferences
conclusions drawn in an indirect way
- –researchers make inferences about the population based on the data gathered from the sample
- indirect because did not study the entire population
- the inference accuracy depends on how well the sample represents the population
what is the most common approach to group research
studying a sample and inferring characteristics of the population as a whole
*value of research depends on how well the sample represents the population
unbiased vs biased sample
- unbiased= all members of the pop have a equal opportunity to be selected
- biased= some members of a pop have an unequal opportunity, or no opportunity, of being selected
bias sources for samples
- failing to identify all members of a population (could be due to differences in the accessible and intended pop or because of a biased sampling method)
- sample of convenience–using a group of participants who are easy to access
- volunteerism–cannot be avoided but because they have to do informed consent everyone who chooses to partake might have a common factor
- **to minimize bias, use different random sampling methods
simple random sampling
every member of the population has an equal chance of being selected for the study
- assign all participants an identifying number, and use a table of random numbers to select participants
- could also use a spreadsheet to do this
- –“RandBeween” function on excel
systematic sampling
generally yields a sample free from intentional bias
- start with a list of potential participants, establish a sampling interval, and select every so many participants according to the number representing your sampling interval
- –the first will be random
stratified random sampling
helps increase the likelihood that the sample accurately represents the population of interest
- –strata or criteria that characterize the pop are identified
- –individuals are chosen at random from each sub-pop (strata)
- –percentage is to be similar to the population ex 50% males and females
cluster sampling
obtain a random sample of predefined groups (med centers, classrooms, etc)
- oftentimes cluster sampling is combined with simple random sampling to create multistage sampling
- –begin with cluster sample list and select a certain # of clusters at random
- –predefined groups or clusters tend to be more similar to one another than the population as a whole, use in studies with relatively large # of patients
purposive sampling
goal is not to generalize findings to a large group, but to obtain expert opinion or personal accounts of people with a unique experience or personal perspectives
- –want to recruit the best source of info
- when the answer to a question requires input from a special persons, must make purposeful effort to id these individuals
evaluating the participant selection procedures of studies
- is difficult if the participants are not thoroughly described, causing the internal validity to suffer
- bad description implies carelessness in sampling
random assignment to groups
required in assigning to experimental and control groups
*can use random number table or spreadsheet
4 reasons why random assignment to groups is important
- adds validity to stats tests
- minimized confounding variables
- randomization while using blinding will eliminate bias
- ethical aspects meaning all participants are treated equally
randomization creates groups that are
- balanced for individual characteristics
- free of experimenter selection bias
- –will be balances when the sample size is 200+
what are randomization options for groups with sample size less than 100
- block randomization
- stratified randomization
- minimization
block randomization
- assigns participants to groups in block
- –block size must be divisible by the number of groups
- –each pattern has an equal chance of being selected with each pattern deciding where the next participants will go
- problem is it is possible to guess some participant allocations so use blinding
stratified randomization
good for small sample size to put into groups
- the smaller the size of the study, the more likely there will be random imbalance between groups
- create a block randomization list for each strata
- impractical to stratify more than 2-3 variables
minimization
- not a method of randomization technically, aims to minimize imbalance
- –1st participant is randomly assigned to a group
- –each following is assigned to a group so that any imbalance between groups is minimized
- advantage is it produced balanced groups and controls for variables
- disadvantage is it violates assumption of randomness, assignment are easily predicted
- use allocation concealment or hide assignment from investigator
power
design sensitivity to detect significance when present
- –poor sensitivity = increased type 2 error
- recommended power is 80%
what is power analysis affected by
- stat test
- internal validity of type I error (a=0.05)
- measurement of reliability (effect size)
how can design sensitivity be calculated
g power test
what are 3 types of single group designs
- single group observed in two or more conditions
- –no scientific comparability
- –no control group
- pre-test/post-test (most common)
- –test sensitization can be a problem)
- –statistical regression
- –appropriate with experimental, hard with quasi experimental
two group designs benefit
allow for control of confounding variables
- involve 2+ groups at different levels of the independent variable
- –one is treatment other is no treatment
- –looking for a treatment effect
parallel design for two group design
participants are assigned randomly to experimental and control groups
cross-over design for two group design
- participants alternate between treatment and control conditions and act as their own controls
- requires smaller number of participants
- suffers from the danger of carryover effects
independent research design for two group design
*requires a random assignment of participants to experimental and control groups
quasi-experimental design
- when it is not possible to assign participants to groups randomly
- –attempt to match groups on the basis or critical variables
- –success depends on how well groups are matched
- –id extraneous variables and match based off of them
experimental design
random assignment of participants to groups
how could you account for extraneous variables in two group designs
- pretest/posttest design
- –problems with sensitization
three ways to increase the complexity of a research design
- add conditions to the independent variable
- add independent variables
- increase the number of groups
multivalent research designs
*two or more levels of the same independent variable
factorial research designs
*2+ independent variables all with different levels
dependent (related) designs
factorial research design where each participant experiences all treatment conditions
independent designs for factorial research
matched groups of participants experience a single treatment condition
mixed designs for factorial research
combination of related and independent variables
interaction with factorial designs
interpretation becomes increasingly difficult as the number of independent variables increase
multiple group designs
- include two or more comparison groups as well as an experiments
- each comparison group controls for one or more variables
- example is 3 groups each with a different degree of HL: mild, moderate, severe
