Lecture 10: effect evaluation Flashcards
What is evaluation?
Determines the degree to which the
campaign reaches its objectives: it helps planners and scholars understand how and why a particular campaign worked and it provides information relevant for planning
future activities (McKee, 2004)
> Making an evaluative judgment
A critical judgment about strong and weak points of campaign and how this can be improved
Did campaign reach objectives?
Why evaluate campaigns?
Knowledge
>Determine / understand campaign effects
>Figure out ‘best practices’ / effective design-principles
>To improve: learn from successes and failures
>Quality (future) meta-analyses depend on quality of individual campaign evaluations (reliable estimate effect sizes)
But also: >Justify costs/resources used >Determine return on investment >Get public/political support >Show successes to secure future funding
2 other important concepts related to effect evaluation
> Efficacy
>Effectiveness
Efficacy
> Campaign evaluated under optimal conditions
Lab research
High internal validity
Low external validity
Usually before campaign launch
Effectiveness
> Campaign evaluation under ‘real life’ conditions
Field research
Difficult (and expensive)
Golden standard
> Randomized Controlled Trial (RCT)
Intervention group vs. control group
Random assignment to groups
Participant and researcher do not know who gets what treatment
Validity
External validity
>Can we generalize the results to other populations, situations and
time periods?
Internal validity
>Can we really attribute results to the intervention/campaign?
Sample
Different independent samples (cross-sectional)?
>Every time different participants
>+ less social desirability
>- long period: changing composition population
Panel design (longitudinal)?
>Every time the same participants
>+ within-participants, changes over time
>- non-response bias due to dropout
>- social desirability (participant wants to report progress)