Post-quiz 4 content Flashcards
Compare the purpose of examining internal and external validity.
External → will the results generalize to the population?
Internal → was the outcome due to exposure, or something else?
What are threats to internal validity?
Name 7.
Factors that can interfere with findings from experimental studies.
- History
- Maturation
- Testing
- Selection
- Instrumentation
- Regression to the mean
- Experimental mortality/attrition
What type of study design is this?
Pre-test, post-test quasi experimental study
(i.e., no randomization or controls)
Describe history as a threat to internal validity.
- Specific events that occurred during the study period.
- External factors that influencing outcome that were not part of the intervention.
- E.g., During the intervention, the school cafeteria started serving more fruits and vegetables.
Describe maturation as a threat to internal validity.
- A function of the passage of time between pre-test and post-test (i.e., people change/grow over time)
- E.g., during the intervention, the students got older and became less picky eaters (they started to like fruits and veggies)
Describe effects of testing as a threat to internal validity.
- Pretesting affects performance on later measurements (post-test)
- E.g., After doing the baseline fruit & veg assessment, the students became more aware of the importance of fruits and veggies and started eating ore of them.
What study design should be used if you were concerned about the effects of testing?
Solomon 4-group
Which threats to internal validity can be reduced by using concurrent groups?
History
Maturation
Testing
What notation correctly describes the study?
A
Describe selection as a threat to internal validity.
- Study groups are not comparable/equivalent
- Selection-history
- Selection-maturation
- Selection-testing
- E.g., schools who received the intervention also have free fruit and veg basket that students can pick from; controls did not have this.
How can selection as a threat to internal validity be controlled?
- Random assignment → if randomization is successful, groups should be similar at baseline.
Describe instrumentation as a threat to internal validity.
- Variability in the analysis and evaluation of the outcomes
- Analytical instruments not calibrated or differently calibrated between experiments.
- Two graders scoring pre/post-tests differently
- E.g., FFQ not validated for study population.
- E.g., 2: Raters assessing post study diet after the nutrition intervention overestimated F&V intake
How can instrumentation as a threat to validity be controlled?
By ensuring measuring instruments are reliable.
(e.g., test-retest, parallel forms, inter/intra rater reliability testing → ensure tools are reliable and give same responses each time to minimize error)
Describe regression to the mean as a threat to internal validity.
- If first values measured are extreme, the post-test values may ‘regress to the mean’ simply as a result of day-to-day variation.
- E.g., by ‘chance’ class A scored very low on pre-test (extreme finding). On second testing, their scores moved towards the mean → may conclude class A improved more when this is actually just ‘regression to the mean’
Describe experimental mortality/attrition as a threat to internal validity.
- Loss to follow-up
- A concern if characteristics of dropouts differ from those who remain in the study, or if dropout rates differ between intervention and control group.
What is intention to treat?
‘once randomized, always analyzed’
Participants are included in analysis in the group they were originally assigned to regardless of whether they completed the trial.
What study design best describes the following:
- 8 elementary schools assigned to receive intervention or not (controls)
- Intervention designed to improve diets, increase physical activity, and decrease screen time
- Assessed students’ diets before and after the intervention
Pre-test, post-test, quasi experimental parallel study
At baseline, the groups did not have different fruit and vegetable intakes.
What threat to internal validity exists here?
Selection
What needs to be considered for external validity?
Need to consider how findings from a study can be generalized to the target population in terms of:
- Sample - representative of population?
- Setting - realistic?
- Study conditions - impact results?
How can external validity be increased?
- Random selection of participants from a population → increase representativeness and ability to generalize results
- Account for the research context → specific aspects of the study may limit the generalizability of the results outside the research settings.
What are advantages of randomized controlled trials? [5]
- Controls for possible confounders → aims to have comparable groups by randomization
- Level of exposure controlled/manipulated by researcher
- Control group for comparison (ensure effects are due to treatment)
- If blinded, reduces bias
- Provides evidence about temporality of associations
What are limits to randomized controlled trials?
- Cost
- Feasibility → ethical implications
- Large sample size needed for statistical power
- Poor compliance and drop outs
- External validity/generalizability
What are 5 nutrition RCT challenges?
-
Duration
- Longer interventions → difficult to have compliance with dietary interventions over long term
- Shorter interventions → long enough to have impact?
-
Controlling exposure
- Difficult to completely ‘control’ intake of a nutrient
- Difficult to have good compliance with nutrition interventions
-
Level of exposure
- Cannot have zero intake group of essential nutrient
- Typically comparing ‘some’ intake with ‘more’ → how to define ‘some’ and ‘more’; ‘some’ may already be enough for outcome of interest → threshold effects
-
Nutrients vs. food
- Will supplement of omega-3 fatty acid produce the same effect as obtaining omega-3 fatty acid from eating fish?
-
Difficult to accurately measure nutrient intake
- Rely on memory and subject’s ability to accurately recall foods consumed & portion sizes
- Foods consumed vary meal-to-meal and day-to-day
- Similar foods may vary in nutrient content.
Thinking critically…
Do omega-3 fats actually have no risk on CVD factors? OR… [5]
- Did subjects already have adequate intakes at baseline?
- Was dose provided clinically relevant?
- Were results influenced by the use of concurrent treatments (e.g., statins)
- Are omega-3 fats only beneficial for prevention of primary CVD not secondary CVD?
- Could effects of omega-3 on CVD risk factors (from observational studies) be explained by something else? (e.g., fish intake, vitamin D and protein, etc.)