Step 2: Design the Study Flashcards
What are the methodologies that may be used for a study?
Quantitative
Focused on measurable outcomes
Testing a hypothesis/theory
Qualitative
Focus on understanding a phenomenon leading to theory development
Mixed
Combination of quantitative and qualitative
What type of methodology focuses on measurable outcomes?
Quantitative
What type of methodology focuses on understanding phenomenon leading to theory development?
Qualitative
Define a population and sample.
How do we know what size of sample to use?
Population
Large group of people who could participate in the research
Sample
Representative of the population
Larger sample size required in quantitative research to enable generalization
Small sample size required in qualitative research; goal is to achieve data saturation
What type of data is collected for each type of study?
Quantitative studies collect numeric data
Qualitative data collects narrative, in-depth data
What are the basic ethical principles that must be considered when designing a study?
o Autonomy, beneficence, non-maleficence, and justice
o Informed consent
o Ethic review boards
What must be demonstrated in quantitative studies that must be considered when designing the study?
Validity
* External – Findings can be generalized to the public
* Internal – did the study measure what it is supposed to measure
Reliability
* Degree of consistency in recording data
What must be demonstrated in qualitative studies that must be considered when designing the study?
REMEMBER: CC DAT to me
Rigour – trustworthiness of the results defined by:
* Credibility
* Confirmability
* Dependability
* Authenticity
* Transferability
How is the researcher views of ontology, epistemology, and axiology shown in their study design?
The methodology they choose, so qualitative, quantitative, or mixed
What influences a researchers decision about what type of methodology to use for a study?
How the researcher views ontology, epistemology and axiology
o Influenced by ways of knowing and the particular paradigm that we attest to
Each influences the methodology that will be used in research to answer the questions of who, what, where, when, why, and how
Asks how do we find out? What strategies are used?
Describe the mixed methodology
What types of approaches of mixed methods did we discuss?
Design study that analyzes both quantitative and qualitative data
Commonly used in program evaluation, organizational studies, and policy development
Specific approaches in this course:
o Sequential - Quantitative followed by qualitative or vise versa
o Concurrent - Quantitative and qualitative at the same time
o Transformative - Want to make some kind of change in the population we are studying
What is the sequential mixed method?
Quantitative followed by qualitative or vise versa
What is the concurrent mixed method?
Quantitative and qualitative at the same time
What is the transformative mixed method?
Want to make some kind of change in the population we are studying
What type of study design uses the positivist/postpositivist paradigm?
Quantitative
What type of study design uses the constructivist/interpretivist paradigm?
Qualitative
What type of study design uses the researcher’s point of view?
Quantitative
What type of study design uses the participant’s point of view?
Qualitative
What type of study design believes that there is only one reality and minimizes context?
Quantitative
What type of study design uses deductive reasoning?
Quantitative
What type of study design studies concepts, constructs, and variables?
Quantitative
What type of study design looks for reliability, validity, and generalizability in their quality of evidence?
Quantitative
What type of study design uses identified concepts are investigated for relationships?
Quantitative
What type of study design tests/explores the relationships among concepts?
Quantitative
What type of study design results in an outcome that accepts or rejects a proposed theory or hypothesis?
Quantitative
What type of study design aims to describe, explain, predict or control?
Quantitative
What type of study design requires a researcher that is neutral and without bias?
Quantitative
What type of study design utilizes a fixed design?
Quantitative
What type of study design looks at a narrowed picture?
Quantitative
What type of study design Uses a contrived setting?
Quantitative
What type of study design collects numerical values, closed-ended data collection, and highly structured writing
Quantitative
What type of study design uses experimental, quasi-experimental, and non-experimental designs?
Quantitative
What type of study design is value-laden?
Qualitative
What type of study design is value-free?
Quantitative
What type of study design values objectivity?
Quantitative
What type of study design values subjectivity?
Qualitative
What type of study design believes that multiple realities exist and in which the context is emphasized?
Qualitative
What type of study design uses inductive reasoning?
Qualitative
What type of study design studies phenomena and concepts?
Qualitative
What type of study design view trustworthiness as the measure of the quality of evidence?
Qualitative
What type of study design identifies concepts or patterns?
Qualitative
What type of study design describes a situation and possibly develops a theory?
Qualitative
What type of study design has an outcome that illuminates a situation or concept or builds a theory?
Qualitative
What type of study design aims to describe or understand?
Qualitative
What type of study design has the researcher involved as an active participant and bias is recognized?
Qualitative
What type of study design has an emerging design?
Qualitative
What type of study design utilizes a complex picture?
Qualitative
What type of study design believes the setting and context are important?
Qualitative
What type of study design has data that includes narrative descriptions, open-ended data collection, and a flexible writing structure?
Qualitative
What type of study design utilizes narrative research, phenomenology, ethnography, grounded theory studies, or case studies?
Qualitative
What type of study design uses sequential, concurrent, or transformative approaches?
Mixed methods
Describe quantitative study designs
- Collecting and analyzing numerical data
- Quantify relationships between variables
- Reality viewed from researcher’s perspective
- Based on the perspective that reality is objective and can be measured
- Focuses on parts rather than wholes
- Targeted to aspects of people’s health-related experiences
- Large number of participants
- Data collection typically precedes data analysis
- Generalizable results to a larger population
- Produces numerical/statistical data
What is the purpose of a quantitative research designs?
Provide a plan of study that permits accurate assessment of relationships between cause-and-effect relationships and between dependent and independent variables
Major concern
o Specify control mechanisms so the answer is clear and valid
Strong design prevents bias from influencing outcomes
Project must be designed so it answers the questions it was supposed to answer so that
o Extraneous factors are controlled
o Degree of generalization to a larger population is valid is valid
What are the critical criteria of a quantitative research study?
Objectivity
o Researcher is at arm’s length from data collection
Accuracy
o Collect accurate data and accurate procedures for analyzing data
Control
o Prevents extraneous variables from influencing outcomes
Feasibility
o Within the timeframe and financial restrictions allotted
o Within restrictions of abilities of the researchers and data collectors
When designing a quantitative research study, what considerations regarding control must be made?
Intervention versus no intervention – determines approach type
Control over the independent variable (IV)
Methods to control extraneous and eliminate confounding variables
o Can these be identified prior to the study?
o Controlling these allows for reliability and validity
A quantitative study may have group comparisons. We discussed 2 of them, what are they?
Between subjects
o Comparing one group of people to another group
Within subjects
o Comparing the same group of people over time
What is a between subjects comparison in a quantitative study?
Comparing one group of people to another group
What is a within subjects comparison in a quantitative study?
Comparing the same group of people over time
What are some examples of settings that may be used for quantitative research studies?
Highly controlled or laboratory
o i.e. petri dishes
Partially controlled
o i.e. hospital setting
Natural or field settings
o i.e. in someone’s home
What are some time frames that may be used when designing a quantitative study?
Cross-sectional - one point in time
Longitudinal - same individuals over a long period of time
Retrospective - looking back in a group to determine ideas about associations for an outcome that has already occurred
Prospective - begins before outcomes, such as at birth, and then watches for outcomes, such as teh development of disease
Describe the cross-sectional timeframe of a quantitative study?
o One point in time
o Might look at people who differ at one point in time
Describe the longitudinal timeframe of a quantitative study?
o Same individuals over an extended period
o Look at variables over months or years
o Used to discover relationships between variables that are not related to various background variables
o Collect data at outset and then periodically over time
Describe the retrospective timeframe of a quantitative study?
o Outcome has already occurred by the time the study started
o Look back in time to formulate ideas about possible associations
o Investigate potential relationships
Correlation not causation
o Might use administrative databases, medical records, interviews with people who already have the condition
o Examines an outcome for which a prospective study is not feasible
Describe the prospective timeframe of a quantitative study?
o Watches for outcomes, such as the development of disease
o Birth cohorts are an example where they collect the group before anyone is ill and then check back in sometimes
Describe the validity of a quantitative study
- To what extent are the findings believable/true?
- Can you rule out other factors as rival explanations?
- The best available approximation of the truth of a given proposition, inference, or conclusion
may be internal or external
Describe internal validity of a quantitative study
- Features of the research context that can compete with the independent variable (IV) to explain what you are observing about the dependent variable (DV) or outcome
- Threats are unreliable findings about the dependent variable
What is external validity of a quantitative study?
- It is the validity of applying the conclusions of a scientific study outside the context of that study
What are threats to an internal validity of a quantitative study? Give some examples
Unreliable findings about the DV
History - external events that are unrelated but occur at the same time as the study
Maturation - natural developmental change
Testing - getting better at a post-test because they did a pre-test
Instrumentation - changes in data collection methods or the instrument to measure the data
Mortality - also called attrition as people drop out or drop dead
Selection bias - poor selection of subjects
Describe a historical threat to the internal validity of a quantitative study
- External events – events that are unrelated and happen outside the study
- They occur at the same time as the IV that can affect the outcome
Describe a maturation threat to the internal validity of a quantitative study
- Developmental change
- Process that occurs within the participants of a study as a result of time
- Most notable in children as their responses will improve and change as they develop
- May happen in new nurses but when you go back they will be improved nurses just because they have been working
Describe a testing threat to the internal validity of a quantitative study
- Testing effects are also called order effects
- Occur in designs that have more than one stage i.e. pre-test and post-test
- Taking a test more than once influences the behaviours and scores on the post-test thus confounding the results
Describe a instrumentation threat to the internal validity of a quantitative study
Reliability of measure
Changes in data collection methods
o Different collectors with slight variations in how they measure
o Poorly trained collectors
o Changing to a BP cuff that hasn’t been calibrated
Confounding by changes in what an instrument measures over time
Describe a mortality threat to the internal validity of a quantitative study
- Sometimes also called attrition
- Participants drop out or drop dead
- Usually recruit more than they need to account for attrition
Describe a selection bias threat to the internal validity of a quantitative study
- Poor selection of subjects
- The way people are chosen to participate in a study
- Arises from differences between groups that exist before the study takes place
- Avoid this with random selection and random assignment into groups
What do the following design notations in a quantitative study mean?
R
X
C
B
A
R – random assignment
X – experimental/treatment group
C – control group or comparison group
B – measurement before the experiment/intervention
o Alternative: O1 – outcome 1, but instructor likes this less
A – measurement after the experiment/intervention
o Alternative: O2 – outcome 2, but instructor likes this less
How is random assignment denoted in a quantitative study?
R
How is the experimental/treatment group denoted in a quantitative study?
X
How is the control group or comparison group denoted in a quantitative study?
C
How is the measurement before the experiment/intervention denoted in a quantitative study?
B
How is the measurement after the experiment/intervention denoted in a quantitative study?
A
Describe the experimental design?
- Involves the manipulation of an independent variable (IV) to observe its effect on a dependent variable (DV) while controlling for other variables
- Classic design has X experimental and C control groups
- Randomization R
(intervention, control, and randomization)
Describe the control group of an experimental design
C
Does not receive experimental treatments
Performance provides a baseline against which the effects of the intervention/treatment can be measured
What types of studies use the experimental design?
After only - test is only done after the intervention
Before and after - testing is done before and after the intervention
Solomon Four Group Design - attempts to control for the effects of pre-testing by adding groups that are not pre-tested
Crossover - Same individuals receive a treatment, have a washout period, and then a second treatment
Factorial - 2 or more IV (factors) are manipulated at the same time
Describe the randomization of an experimental design
R – subjects are randomly assigned to groups
Critical characteristic of experimental design
Randomly assigned to experimental or control groups
Compares outcomes between the two groups
Allows establishment of cause and effect or correlational relationships
Randomization
Different from random selection – how you select the sample
Randomization refers to the assignment to groups
What is a critical characteristic of the experimental design?
Randomization
Describe the after only design
A type of experimental design
Also called a post-test only design
Random assignment to 2 groups
o R X A
o R C A
Equivalent groups
o Randomization spreads the variation of sample between the two groups
What type of design has the following two groups:
R X A
R C A
After only design, which is an experimental design
Describe the before-after design
A type of experimental design
Also called equivalent groups or pre-test/post-test designs
Random assignment to two groups
o R B X A
o R B C A
Test is done before and after the intervention
Disadvantage
o Using the same test/observation, people improve when they do them a second time which can confuse the results
(testing threat to validity)
What is a disadvantage to the before-after test design?
This is a type of experimental design that test people before and after the intervention.
The issues is that using the same test/observation, people improve when they do them a second time which can confuse the results
(testing threat to validity)
What type of design has the following two groups:
R B X A
R B C A
Before-after design, which is an experimental design
Describe the Solomon Four group design?
What is a disadvantage of this design type?
A type of experimental design
Attempts to control for the effects of pre-testing by adding groups that are not pre-tested
A combination of the After-only and Pre-test/Post-test designs
Format
o R B X A
o R B C A
o R X A – so no pre-testing has been done for this group
o R C A – so no pre-testing has been done for this group
The 2 X and 2 C groups can be contrasted to verify the difference in the post-test as a result of the pre-test
Disadvantage
o Requires a large sample group so it has a higher cost
What type of design has the following groups:
R B X A
R B C A
R X A
R C A
Solomon four group design, a type of experimental design
Describe the crossover design
A type of experimental design
- Also called repeated-measures design
- Considered experimental if subjects are randomized to treatments
- Researcher exposes the same study participants to more than one intervention
- Each subject serves as their own control by crossing over from one treatment to another
- For example, a patient receives treatment A, then have their condition assessed during a washout period, then they are given treatment B and then their condition is reassessed
What type of design has the following groups:
R T1 A W T2 A
R T2 A W T1 A
W - washout period
T1 - treatment 1
T2 - treatment 2
Crossover design