Evaluation design I Flashcards
stages of evaluation
formative
process
outcome
The main purpose of evaluation design is to be as confidence as poss that any observed changes were cause by intervention, rather than by chance/other unknown factors
formative eval
Before intervention
Acceptability and feasibility of intervention
Mainly qual, e.g. focus groups, in depth interviews
formative eval research
Penn et al. (2018)
Penn et al. (2018)
NHS diabetes prevention programme
eval - qual research
behav interventions
specification reflected evidence - framework for service provision
provides ev based behav intervention for prevention of T2D in high risk adults
process eval
Measures how intervention delivered and received
Mixed qual and quan
Done along the way - make alterations if necessary
process eval research
Sanchez et al. (2017)
Sanchez et al. (2017)
improve understanding of underlying mechanisms that may impact results
prescribe healthy life intervention
moderate–>good performance on adoption, reach and implementation
outcome eval
Measures whether intervention achieved objectives
Mainly quan
outcome eval research
Ebert et al. (2018)
Ebert et al. (2018)
internet and mobile stress management intervention and RCT
intervention v control
int = 7 sessions of problem solving and emotion regulation techniques
baseline v 6 months
cost-effective and lead to cost savings
stages of evaluation research
Dehar et al. (1993)
Nutbeam (1998)
Dehar et al. (1993)
formative - develop and improve programmes at an early stage
process - info on programme implementation, interpretation of outcomes and guiding future research
Nutbeam (1998)
issues with definition and measurement of outcomes and use of eval methods
most powerful interventions = LT
technical problems from scientific rigour and advantages of less well defined content
combine and quan and qual
evals tailored to intervention
cause and effect
Want to determine whether a cause-effect r’ship exists between intervention and outcome
logic of causal inference
Under what conditions may we infer that a change in the DV (PA) was really caused by IV (intervention) and not by something else (envs etc.)
What are some of the most plausible rival explanations, and how do we rule them out?
Logic of causal inference research
Rothman and Greenland (2004)
Hill (1975)
Rothman and Greenland (2004)
causality often debated
more general conceptual model required
Hill (1975)
https://journals.sagepub.com/doi/pdf/10.1177/003591576505800503)
criteria for inferring causality
temporal r’ship
plausibility
strength of association
dose-response r’ship
reversibility
temporal r’ship
Cause (intervention)must precede effect (increase in PA)
plausibility
Association plausible, and more likely to be causal, if consistent with other knowledge
E.g. reasonable to expect people who receive exercise intervention will increase PA by greater amount than people without intervention
strength of association
Strong association, as measured by effect size or relative risk, more likely to be causal than weak association
