Key Sources Flashcards
Petticrew, 2011
Moderator analyses carry important implications for the ‘equity’ of social intervention effects; key topic in public health
Richard Peto
Expressed misgivings about moderator analyses in trials:
o “Only one thing is worse than doing subgroup analyses – believing the results”
Rothwell, 2005; Wang and Ware, 2013
To prevent ‘cherry picking’ of results we need explicit pre-specification of hypotheses—confirmatory or exploratory—plus rationale
Thompson and Higgins, 2005
Limitations of analyzing moderators in systematic reviews:
o Power still low: although total Ns may be higher, subgroups are coded at trial level, meaning all variability within trials of participant characteristics is masked
Lipsey, 2003
Limitations of analyzing moderators in systematic reviews:
o Moderators still confounded
Leijten et al., 2013
Examined how socioeconomic status (SES) moderator effects were confounded by other risk factors, such as problem severity, attempting to overcome the issues surrounding confounding of moderators that are common to moderator analyses
Bonell et al., 2012
We can derive mediator hypotheses from theory, or from qualitative methods; e.g., users’ views on process
The importance of theory in evaluation:
o RCTs provide relatively simplistic tests of theories – controlled design requires relatively few assumptions (inputs, outcomes)
o This approach to evidence generation is often orientated towards accreditation of interventions rather than tests of causal theories
o But for social interventions causal pathways may not be straightforward and allocation to interventions may be uncontrolled
o We need to know how interventions work
Moore et al., 2014
Importance of process evaluations:
o “In order for evaluations to inform policy and practice, emphasis is needed not only on whether interventions ‘worked’ but on how they were implemented, their causal mechanisms, and how effects differed from one context to another”
‘Black box’ critique:
o “…the reader is left with data on whether or not an intervention works, but little insight into what the intervention is”
Freedman, 1987
Equipoise (a key feature of RCTs): Genuine uncertainty about the relative merits of the treatments being compared; unethical otherwise
Drummond et al., 2005
Naming alternatives and explicitly considering them in an economic evaluation:
o …what are the alternatives?
o …what is the perspective?
o …what does the economic evaluation tell us that an ‘educated guess’ won’t?
McCord, 2003
In her examination of the negative outcomes of the Cambridge-Somerville Youth Study and a host of other crime prevention programmes shown to ultimately cause harm, Joan McCord advocates an approach to assessing social interventions that takes into account more than mere efficacy, also looking into the safety measures and possible iatrogenic effects of a programme, her aim being to demonstrate that simply asking whether an intervention ‘works’ fails to adequately capture crucial considerations related to propensity for harm
In large part due to its uniquely sustained efforts in keeping records on the life outcomes of its participants, the Cambridge-Somerville Youth Study occupies a foundational role in the ever-expanding literature on the subject of social interventions that harm
However, the continued impact of the Cambridge-Somerville Study cannot solely be attributed to the shocking findings of its follow-up inquiries, which determined that those boys at risk of juvenile delinquency assigned to the treatment group died an average of five years earlier than their counterparts in the control group, and were at greater risk of eventually receiving a serious mental health diagnosis
Rather, this study continues to wield such influence because it so clearly exemplifies the increasingly recognized fact that conscientious study design, sufficient funding, proper execution, and the best of intentions do not necessarily culminate in an ‘effective’ intervention, or act as sufficient deterrents against undesirable outcomes—the potential for harm must be considered from the outset, and any hypothesized ‘harm-inducing’ mechanisms scrupulously monitored throughout the duration of the intervention
Merton, 1936
Alongside limitations in the existing state of knowledge, ignorance constitutes a driving factor of ‘unexpected consequences of conduct’ in purposive social action, as defined by Robert K. Merton
Lorenc, 2014
Theo Lorenc (2014) proposes the following typology of harm:
o Direct harms, in which the outcomes desired are directly associated with adverse effects
o Psychological harms, in which an intervention yields negative mental health impacts
o Equity harms, in which an intervention worsens existing social inequalities
o Group and social harms, in which harm is generated by the singling out or bringing together of a certain group
o Opportunity harms, in which, by favoring a particular intervention over others, we forfeit claim to any potential benefits associated with alternative courses of action
Cook, 2000
“It should be possible to construct and justify theory-based form of evaluation that complements experiments…It would prompt experimenters to be more thoughtful about how they conceptualise, measure, and analyse intervening processes. It would also remind them of the need to first probe whether an intervention leads to changes in each of the theoretically specified intervening processes…”
Cook and Campbell, 1979
“From Popper’s work, we recognize the necessity to proceed less by seeking to confirm theoretical predictions about causal connections than by seeking to falsify them. For Popper, the process of falsification requires putting our theories into competition with each other”
Typology of validity:
Enumerates four distinct yet complementary components of validity to be heeded in social research settings: (1) internal validity, (2) external validity, (3) construct validity, and (4) statistical conclusion validity
Martinson, 1974
In the absence of theories: null findings
o “What works? Questions and answers about prison reform” The Public Interest, 35, 22-54
o Review of rehabilitative interventions for reducing recidivism
o Widely interpreted as “nothing works” in prison rehabilitation
o Led some to criticize the investment of resources in prisoner rehabilitation
o But led others to criticize the methodological status quo and ask why rehabilitative programmes don’t work
Coryn et al., 2011
Definition #1 of theory-based evaluation:
o “Theory-driven evaluation is…any evaluation strategy or approach that explicitly integrates and uses stakeholder, social science, some combination of, or other types of theories in conceptualizing, designing, conducting, interpreting, and applying an evaluation”
Weiss, 2000
Definition #2 of theory-based evaluation:
o “It helps to specify not only the what of a programme outcomes but also the how and the why. Theory-based evaluation tests the links between what programmes assume their activities are accomplishing and what actually happens at each step along the way”
Stockwell and Gruenewald, 2004
Theory-based evaluation example: Limiting the physical availability of alcohol to reduce alcohol-related harm
o “Efforts to control alcohol availability to reduce alcohol-related harms have been based on the view that ‘less is best’; i.e. the less alcohol available the better for public health and safety”
“Availability theory” – 3 related propositions:
- (1) The greater the availability of alcohol, the higher the average consumption of alcohol
- (2) The higher the average consumption, the greater the number of excessive drinkers
- (3) The greater the number of excessive drinkers, the greater the prevalence of health and social problems
Prevention might include:
- Placing restrictions on number of premises (i.e., spatial availability)
- Placing restrictions on the times at which alcohol can be sold (i.e., temporal availability)
- Placing restrictions on consumption for population groups (e.g., minimum legal drinking age)
Humphreys and Eisner, 2014
Found no real effect of the Licensing Act (2003) in achieving its aims – the Act did not cause bars and nightclubs to change their trading hours a great deal
Craig et al., 2008
“Process evaluations, which explore the way in which the intervention under study is implemented, can provide valuable insight into why an intervention fails or has unexpected consequences, or why a successful intervention works and how it can be optimised. A process evaluation nested inside a trial can be used to assess fidelity and quality of implementation, clarify causal mechanisms, and identify contextual factors associated with variation in outcomes”
Updated guidance from the Medical Research Council (MRC) emphasizes the importance of conducting process evaluations within intervention trials, stating these evaluations “can be used to assess fidelity and quality of implementation, clarify causal mechanisms and identify contextual factors associated with variation in outcomes”
“Best practice is to develop interventions systematically, using the best available evidence and appropriate theory, then to test them using a carefully phased approach, starting with a series of pilot studies targeted at each of the key uncertainties in the design, and moving on to an exploratory and then a definitive evaluation”
Moore et al., 2015
“An intervention may have limited effects either because of weaknesses in its design or because it is not properly implemented. On the other hand, positive outcomes can sometimes be achieved even when an intervention was not delivered fully as intended. Hence, to begin to enable conclusions about what works, process evaluation will usually aim to capture fidelity (whether the intervention was delivered as intended) and dose (the quantity of intervention implemented). Complex interventions usually undergo some tailoring when implemented in different contexts. Capturing what is delivered in practice, with close reference to the theory of the intervention, can enable evaluators to distinguish between adaptations to make the intervention fit different contexts and changes that undermine intervention fidelity”
“Complex interventions work by introducing mechanisms that are sufficiently suited to their context to produce change, while causes of problems targeted by interventions may differ from one context to another. Understanding context is therefore critical in interpreting the findings of a specific evaluation and generalising beyond it. Even where an intervention itself is relatively simple, its interaction with its context may still be highly complex”
Baron and Kenny, 1986
VALUE OF MEDIATOR ANALYSES
o Operating under the basic assumption that a causal relationship exists between an intervention and an observed outcome—an assumption made reasonable in the context of RCTs by the process of randomization—we can conceptually frame a mediator as an intervening variable on the causal pathway of an intervention, responsible for somehow shaping the relationship between stimulus and response
Kraemer, Wilson, Fairburn, and Agras, 2002
VALUE OF MEDIATOR ANALYSES
o By striving to fully appreciate “the mechanisms through which treatments operate,” we find ourselves better equipped to maximize treatment effectiveness, and simultaneously more able to reduce the monetary and human costs associated with a given treatment—“Active therapeutic components could be intensified and refined, whereas inactive or redundant elements could be discarded”
Clark, 1997
VALUE OF MEDIATOR ANALYSES
o Notably, when evidence emerged to suggest that cognitive behavioral therapy (CBT) works in treating panic disorders via the eradication of catastrophic thoughts related to bodily changes—a mediating mechanism—the cognitive theory of panic gained greater empirical substantiation
Lorenc et al., 2013
VALUE OF MODERATOR ANALYSES
o Famously, a preponderance of evidence strongly suggests that media campaigns targeted at reducing cigarette use prove most effective at achieving their aim among the socioeconomically well-off, thereby widening already established inequalities
Supplee et al., 2013
VALUE OF MODERATOR ANALYSES
o Moderator analyses also present a means by which to capitalize on the increasing interest among policymakers in promoting more targeted, highly tailored interventions, with the initial investment incurred in developing a well-founded knowledge base of what works for whom seen as more than justified by the resulting ability to maximize efficiency and minimize risk
o The considerable influence wielded by intervention research pertaining to the topic of subgroup analysis extends to “policy decisions around programmatic aims (e.g., Upward Bound), funding decisions (e.g., Even Start), and new initiatives targeting funding towards evidence-based programs (e.g., teen pregnancy and home visitation)”
EXAMPLE: Studio Schools
What are Studio Schools (briefly)?
o New model of education in England
o Aim to contextualize learning and make it more practical
o Outcomes are engagement with education and employability of young people (14-19 years old)
Specifying the components of Studio Schools o Core: - Project-based learning - Personal coaching sessions - Work placements - Small school environments - Longer school day and year o Allowable: - Opportunities to start a business or project - Self-study units - Taught subject lessons
EXAMPLE: WHO ‘Parenting for Lifelong Health’ - Parenting interventions to reduce risk of child maltreatment in low and middle income countries (LMICs)
South Africa: The Sinovuyo Caring Families Programme for Parents of Children Aged 2-9 Years
To test feasibility using mixed methods:
- Dosage/exposure
- Programme fidelity
- Participant satisfaction, cultural feasibility
- Pilot RCT
Pilot program delivered to 56 parents in 4 groups
Parent interviews at participants’ home:
- Random sample (intervention, n=11; control, n=4)
- 1 hour; trained research assistants with interpreter
Facilitator focus groups at center
- 2.5 hours, conducted by Lachman in English
- Post-program (intervention, n=8; control, n=6)
Interview protocols with open-ended approach:
- Acceptability of program content, delivery methods
- Changes observed at home by parents
- Training, supervision, and logistical support
Kaminski et al., 2008
IDENTIFYING CORE COMPONENTS
Meta-regression: linking components to intervention effect sizes
o Interventions that DO include component X versus interventions that DO NOT include component X
o Strengths:
- Based on dozens of studies
- Results do not hinge on single intervention ‘brand’ or trial
- Can include many different types of components
o Limitations:
- No causality: only association between components and outcomes
- Results depend on patterns of combinations of components in existing programmes
Collins, 2016
IDENTIFYING CORE COMPONENTS
Factorial experiments
o “The process of identifying the intervention that provides the highest expected level of effectiveness obtainable…Within key constraints imposed by the need for efficiency, economy, and/or scalability.”
Dwan et al., 2008
Critical appraisal: what limitations of moderator analyses in RCTs?
o Cherry picking – evidence that reporting bias common in main effect analyses of trials – more so when it comes to secondary analyses?
The continued prevalence of outcome reporting bias in primary (i.e., main effect) analyses of trials, as highlighted by the work of Dr. Kerry Dwan and colleagues (2008), raises concerns that such ‘cherry picking’ of favorable results may abound to an even greater extent in secondary analyses of mediators and moderators
Assmann, Pocock, Enos, and Kasten, 2000
Critical appraisal: what limitations of moderator analyses in RCTs?
o Note with apprehension the prohibitively low statistical power of a great deal of secondary analyses, finding that “[m]any [major clinical trial] reports put too much emphasis on subgroup analyses that commonly lacked statistical power”
Brown et al., 2013
Pooling data
o Scientific benefits of sharing, collaboration between many investigator teams = better science
o Climate now is right: big push from funders, journals, governments to share data to increase transparency, reduce fraud (NIH, Ben Goldacre, BMJ AllTrials campaign)
o Example: NIMH Collaborative Data synthesis for Adolescent Depression Trials
Pooling data from a number of trials can address the lack of statistical power common to secondary analyses, while also providing for greater generalizability across contexts
Blankenship et al., 2006
What are structural interventions?
o “Structural interventions differ from many public health [and social] interventions in that they locate, often implicitly, the cause of public health [and social] problems in contextual or environmental factors that influence risk behavior […] rather than in characteristics of individuals who engage in risk behaviors”
o Structural interventions aim to change “social, economic, political or physical environments”
o This is different from approaches that focus on the individual, because the underlying assumption is that people’s context constrains their autonomy and affects how they make choices and act on those decisions
Blankenship et al., 2000
A framework when thinking about structural interventions: o Three types of contextual factors: 1. Availability 2. Acceptability 3. Accessibility
o Three levels at which structural interventions are targeted:
- Individual
- Organizational
- Environmental
Availability
o Focus on “behaviors, tools, equipment, materials, or settings that are necessary”
o Can address a lack of beneficial resources or an abundance of harmful resources
Acceptability
o Changing (altering) social norms
o Risk is “[…] partially determined by [society’s] values, culture and beliefs, or those of subgroups within it”
Accessibility
o Availability does not necessitate accessibility: “[…] the ability to avail oneself of [tools etc.] may be restricted by lack of resources and power”
o “[…] is a function of social, economic and political power and resources”
Examples of structural interventions for HIV:
- Comprehensive sex education with access to male and female condoms
- Syringe exchange programs
- Healthcare availability
- Stable housing
Bronfenbrenner, 1977
Microsystem
o The relationship between the individual and their proximal environment (“immediate setting”)
o “(e.g., home, school, workplace, etc.). […] The factors of place, time, physical features, activity, participant, and role constitute the elements of a setting”
Mesosystem
o The relationships between the main environments that an individual interacts/lives in
o “for an American 12-year-old, the mesosystem typically encompasses interactions among family, school, and peer group; for some children, it might also include church, camp, […] In sum, stated succinctly, a mesosystem is a system of microsystems”
Exosystem
o Extension of the mesosystem and does not contain the individual
o It is the social structures around the individual and other lower levels that influence the environment (“immediate settings”)
o “These […] encompass, among other structures, the world of work, the neighborhood, the mass media, agencies of government (local, state, and national), the distribution of goods and services, communication and transportation facilities, and informal social networks”
Macrosystem
o “A macrosystem refers to the overarching institutional patterns of the culture or subculture, such as the economic, social, educational, legal, and political systems, of which micro-, meso-, and exo-systems are the concrete manifestations”
“[…] environmental structures, and the processes taking place within and between them, must be viewed as interdependent and must be analyzed in systems terms.”
Adimora and Auerbach, 2010
SOCIAL DETERMINANTS OF HEALTH – HIV IN THE U.S.
53% of new HIV infections occur in men who are gay and men who have sex with men (MSM)
Homophobia and negative attitudes towards MSM are evident in the political and legal context:
o Policies on sexual behaviours (e.g., sodomy)
o Policies on relationships (e.g., marriage)
o “These restrictions tend to marginalize and exclude gay people and drive their relationships underground. Thus, many MSM do not publicly identify (or self identify) as ‘gay,’ or seek HIV prevention and sexual health information services targeted to gay communities. Internalized homonegativity has been associated with unprotected anal intercourse, a major route of HIV transmission, particularly for gay and other MSM”
~12% of new HIV infections occur among people who inject drugs:
o Lack of access to sterile needles and syringes
o Lack of access to addiction treatment programmes:
- Until Jan 2010, ban on federal funds being used to support syringe exchange programmes
- Political and ideological factors – criminalizing drug use, ‘war on drugs’
HIV prevalence higher among people who are poor:
o Lack of access to healthcare
o Increased risk of exposure to crack cocaine
o Risk of transactional sex for drugs
o Unstable housing, homelessness = increased risky behaviours
Fraser, 2009
Developing theory – sometimes called ‘problem theory’
o “Problem theory is a portrayal of the individual and environmental factors – both risk inducing and risk suppressing (i.e. protective) – that give rise to a problem or that sustain a problem over time. We use problem theory to identify leverage points for intervention”
o Incorporates risk and protective factors
Weiss, 1995
The role of theories in policy and practice
o “In a sense, all policy is theory. A policy says: if we do A, then B (the desired outcomes) will occur. As evaluative evidence piles up confirming or disconfirming such theories, it can influence the way people think about issues, what they see as problematic, and where they choose to place their bets. The climate of opinion can veer and wiser policies and programs become possible”
Lipsey, 1993
What had gone wrong in prisoner rehabilitation research?
o Adherence to particular methods stifled creativity in developing appropriate methods
o Excessive use of black-box methods underrepresents complexity
Medical Research Council (MRC) Framework
MRC Framework, stage 1: Development of the intervention
o Identifying the evidence base – background lit on nature, prevalence, risk and protective factors, intervention effects (use/do SRs)
o Identifying or developing theory [underlying the problem]
o Modelling process and outcomes [for the intervention]
MRC Framework, stage 2: Feasibility and piloting
o Testing procedures [for the intervention and all research processes]
o Estimating likely resources, recruitment and retention
o Participant acceptability and satisfaction
o May include pre-post study, or small RCT
o Can be qualitative and/or quantitative
MRC Framework, stage 3: Evaluation
o Assessing effectiveness
o Understanding change process
o Assessing cost effectiveness
MRC Framework, stage 4: Implementation
o Dissemination
o Surveillance and monitoring
o Long term follow-up
Hutchings, Gardner, and Lane, 2004
Key components of parenting interventions:
o Social learning through rehearsal of skills
o Principle-based rather than techniques
o Home practice of parenting skills
o Developmentally appropriate activities
o Positive parenting combined with non-violent discipline
o Child-directed play
o Effective supervision and parent-child communication
Donner, 1998
Cluster randomised controlled trials (cRCTs) are:
o ‘[…] experiments in which intact social units, rather than independent individuals, are randomly assigned to intervention groups’
Dahlberg et al., 2002
The Centers for Disease Control and Prevention (CDC) uses a four-level social-ecological model to better understand violence and the effect of potential prevention strategies:
o This model considers the complex interplay between (1) individual, (2) relationship, (3) community, and (4) societal factors
o It allows us to understand the range of factors that put people at risk for violence or protect them from experiencing or perpetrating violence
o The overlapping rings in the model illustrate how factors at one level influence factors at another level
o Besides helping to clarifying these factors, the model also suggests that in order to prevent violence, it is necessary to act across multiple levels of the model at the same time – this approach is more likely to sustain prevention efforts over time than any single intervention
(1) Individual
o The first level identifies biological and personal history factors that increase the likelihood of becoming a victim or perpetrator of violence
o Some of these factors are age, education, income, substance use, or history of abuse
o Prevention strategies at this level promote attitudes, beliefs, and behaviors that prevent violence
o Specific approaches may include education and life skills training
(2) Relationship
o The second level examines close relationships that may increase the risk of experiencing violence as a victim or perpetrator
o A person’s closest social circle (peers, partners and family members) influences their behavior and contributes to their experience
o Prevention strategies at this level may include parenting or family-focused prevention programs, and mentoring and peer programs designed to reduce conflict, foster problem solving skills, and promote healthy relationships
(3) Community
o The third level explores the settings, such as schools, workplaces, and neighborhoods, in which social relationships occur and seeks to identify the characteristics of these settings that are associated with becoming victims or perpetrators of violence
o Prevention strategies at this level impact the social and physical environment – for example, by reducing social isolation, improving economic and housing opportunities in neighborhoods, as well as the climate, processes, and policies within school and workplace settings
(4) Societal
o The fourth level looks at the broad societal factors that help create a climate in which violence is encouraged or inhibited
o These factors include social and cultural norms that support violence as an acceptable way to resolve conflicts
o Other large societal factors include the health, economic, educational and social policies that help to maintain economic or social inequalities between groups in society
Fanslow and Robinson, 2010
Intimate Partner Violence (IPV) reporting
o More severe physical violence more likely to be reported
o Physical violence more likely than sexual to be reported
How many and who is most affected?
o Official data:
- Police reports, hospital visits
o Survey data:
- Surveys, interviews
o Measurement:
- General/loaded terms underestimate violence
- Measurement debates represent broader theoretical divide:
1. IPV is a type of criminal or family violence, equally a problem among men and women
2. IPV is part of a broader group of violence acts perpetrated against women, within a social context that values men over women
Do these differences in measurement reflect different typologies of IPV? o Situational couple violence o Intimate terrorism o Violent resistance o Mutual violent control
IPV against women
o Overall prevalence (physical/sexual) over 30%
o The U.S. experiences the highest rates of intimate partner violence worldwide
Baldwin and Lindsley, 1994
Culture is a complex and multidimensional construct
o Over 100 varied definitions of culture
o Can be differentiated from “race” and “ethnicity” (but sometimes terms are used interchangeably)
o Race tends to assume a biological basis can be applied to classifying humans, yet there is more genetic variation within races than between races; race more of a social construct than a biological one
o Ethnicity attempts to further differentiate racial groups according to lineage (e.g., country of origin, location, religion, language); like race, it carries its own historical, political and social baggage
o Knowing someone’s ethnic identity or national origin does not reliably predict beliefs and attitudes
Bernal et al., 2009
Cultural adaptation
o “The systematic modification of an evidence-based treatment or intervention protocol to consider language, culture and context in such a way that it is compatible with the client’s cultural patterns, meanings and values”
Lau, 2006
WHY IS CULTURAL ADAPTATION IMPORTANT?
Ecological validity argument
o Much of the research supporting EBIs is said to have been conducted in white middle-class populations or in high-income countries; findings may not generalise to other groups (which EBIs?)
o Poor acceptability of “evidence-based” or ‘imported’ approaches shown in some communities; e.g., high dropout
Griner and Smith, 2006
WHY IS CULTURAL ADAPTATION IMPORTANT?
Evidentiary argument
o Strong external validity: Meta-analysis of culturally adapted mental health interventions; effect sizes (d = 0.45) suggest moderately strong benefit of culturally adapted interventions
o However – lacks counterfactual – makes no comparison with non-adapted programs
o Interventions targeted to a specific cultural group were more effective than those for mixed groups
o NOTE: conflicting evidence, e.g., Huey et al., 2008; 2012; Gardner et al., 2016
Baumann et al., 2015
Review of the types/functions of cultural adaptation frameworks
Bernal et al., 1995
ADAPTATION FRAMEWORKS
Types/functions of cultural adaptation frameworks:
o To inform modification to content of the intervention
- e.g., Ecological Validity Model
Ecological Validity Model
o Originally developed with Latino populations
o Identifies 8 “culturally sensitive” elements for adaptation:
- (1) Language: Does the user understand language, idioms and words used?
- (2) Persons: Is the user comfortable with the similarity (or difference) in the cultural identity of the provider?
- (3) Metaphors: Are cultural symbols and concepts used appropriately?
- (4) Content: Does the user feel that the intervention acknowledges his/her cultural values and traditions?
- (5) Concepts: Are intervention concepts framed within cultural context? Is the user in agreement with problem definition and intervention rationale?
- (6) Goals: Are intervention goals framed within adaptive cultural values of the patient? Does the user agree with the goals of treatment?
- (7) Methods: Do intervention methods fit with the user’s expectations and preferences?
- (8) Context: Does the intervention consider contextual issues such as migration and acculturation stress, social supports, family relationships and access barriers?
Ferrer-Wreder et al., 2012; Backer, 2002
ADAPTATION FRAMEWORKS
Types/functions of cultural adaptation frameworks:
o To inform the process of adaptation
- e.g., decisions about when to adapt, how to adapt, and which stakeholders should be involved
Domenech-Rodríguez and Wieling, 2004
Cultural adaptation is a relatively new development in intervention science
o Aim to “strike a balance between community needs (for adaptation) and scientific integrity” (incorporating EB principles and fidelity to these)
Gottfredson, 2002
ARGUMENTS AGAINST CULTURAL ADAPTATION: Some studies hint at higher retention but lower effectiveness of cultural adaptations
Gottfredson SFP trial in DC: Adaptations helped engagement, but intervention was not effective – tricky to achieve balance between cultural/community adaptation and fidelity
Barrera et al., 2013
Adaptation methods
o Cultural Adaptations of Behavioral Health Interventions: A Progress Report
Surface Structure (or “Presentation Strategies”)
- Bilingual and bicultural materials and staff
- Translation/back-translation of materials
- Inclusion of lifestyle elements, such as food, music
- Delivery of the intervention in group settings
- Situating the intervention in safe settings familiar to participants
- Incorporation of culturally familiar formats and activities
- Inclusion of culturally important role models
Deep Structure (or “Content Strategies”)
- Incorporation of cultural values in intervention design or implementation (e.g., ‘time-out’ in Norway)
- Involvement of the family in interventions
- Adjustment of materials to literacy level of participants
- Use of social support and networks
Cultural adaptation doesn’t entail completely discarding programme theory / theory of change—instead, uses theory and procedures from initial efficacy trials as a base, but informs that knowledge with input from the subcultural group being targeted
Adaptation is more participatory; accounts more for target populations and creates formal space for their input
o Can build greater trust with the community
o One specific form of cultural adaptation is “constituent-involving”; i.e., drawing on cultural knowledge and experience
HOWEVER: Potentially difficult to directly compare cultural adaptation to original intervention—comparison to SoC [standard of care] is the norm, but that might be an inaccurate comparison
o Using finite resources for adaptation, while not knowing if the intervention will work/be beneficial, risks increasing harm and/or ineffectiveness
Vellakkal and Patel, 2015
Adaptation methods
o Program for Effective Mental Health Interventions in Under-resourced Health Systems (PREMIUM)
CASE EXAMPLE: Developing a brief parenting psychoeducation intervention in Burundi (Jordans et al., 2011)
Setting: Central African nation affected by civil war along ethnic lines; resulted in 250,000-300,000 deaths and 880,000 displaced people
Peace accords in 2003, but high levels of poverty and “damaged social and moral fabric due to violence” act as main risk factors for mental health problems
Project involved (i) mapping and selection of existing local and global intervention components based on expert and community stakeholder perspectives, and (ii) controlled evaluation
Phase 1 – Assessing needs (qualitative study)
o Explored:
- (i) Necessity and relevance of family-based care
- (ii) Stakeholder views of and recommendations for intervention content and delivery methods
o Data collection methods (used local researchers):
- Focus group discussions with children, teachers, parents, facilitators of existing psychosocial programmes (n=99)
- Key informant interviews with clergy, healers (n=11)
- Group discussions with parents around specific case vignettes related to milestones, discipline and poverty (n=68)
- Semi-structured interviews with children involved in existing programmes, and parents (n=25)
o “Community sensitisation meetings” used for initial purposive sampling, followed by snowball sampling
Phase 2 – Identifying and prioritizing types of interventions (lit review and expert consultation)
o Aim: to generate field-tested intervention modalities in LMIC settings
- Expert panel assembled to further generate and rank intervention options
- Followed existing criteria and guidelines for priority setting (Child Health and Nutrition Research Initiative)
- Process involved (i) gathering technical experts, (ii) systematic listing of intervention options, and (iii) scoring options by pre-defined criteria
- Participants selected based on experience in psychosocial and mental health work for children in LMIC
- Expert panel (n=60): first round further generation of options; second round prioritization based on set of criteria (Acceptability; Feasibility; Effectiveness)
Phase 3 – Identifying specific elements of intervention (systematic review)
o Reviewed academic literature to determine common practice elements of evidence-based interventions for conduct and mood problems
o Drew on existing database that distilled evidence-based interventions into constituent elements—context and delivery
Phase 4: Assessing acceptability and feasibility of identified components (stakeholder assessment)
o Stakeholder assessments conducted to assess cultural acceptability and feasibility of all components generated in previous steps
o Respondents included care manager, service providers and potential beneficiaries (n=40)
Results—Stakeholder assessment
o Strong endorsement for selected components
o In addition, stakeholders provided suggestions on how to implement the intervention strategies:
- e.g., respondents recommended training teachers and community workers in addition to health workers, follow-up through home-visits, and reinforcing family ceremonies as a way of strengthening networks
o While acceptance was high, respondents commented that it was difficult for parents to give time because of the loss of productivity that comes with joining interventions
o Respondents therefore emphasised the need to start with raising parental awareness and creating willingness to participate in interventions…
o …then adaptation and finalisation of intervention protocol
Summary and conclusions
o One size does not fit all
- Through cultural adaptations it may be possible to go beyond the one-size-fits-all approach and modify interventions at multiple levels (e.g., deep vs. surface; 8 ecologically valid domains)
- Or some interventions may be culturally flexible, so the adaption happens at the level of each participant (see Barrera et al., 2013; also Incredible Years parenting a possible example)
o Community engagement is important across all phases of cultural adaptation, from needs-assessment through to implementation
o Potential for cultural adaptations to enhance engagement, outcomes and sustainability of interventions, but issues of fidelity require careful attention
Gardner et al., 2016; Reid et al., 2001; Leijten et al., 2016
Do adapted interventions work better?
o Some interventions appear to ‘transport’ well across cultures, despite limited adaptation (in parenting field – Gardner et al., 2016; Reid et al., 2001; Leijten et al., 2016 compared home-grown and transported programs, finding no difference in effects)
Kumpfer et al., 2002
ARGUMENTS AGAINST CULTURAL ADAPTATION: Some studies hint at higher retention but lower effectiveness of cultural adaptations
Do adapted interventions work better?
o Culturally adapted family-based interventions for children can increase retention by up to 40%, but also reduce positive outcomes – suggests a possible trade-off between acceptability and effectiveness
Mihalic, 2002
Defining implementation fidelity:
o The rigor of sticking to the original intervention as it was designed and tested; degree to which program implemented as intended by developers
o Implementation fidelity = adherence = integrity = determination of how well a program is being implemented in comparison with the original program designed
5 components of implementation fidelity:
o (1) Adherence = whether intervention is being delivered as designed, with all core components delivered to appropriate population by trained staff using the right protocols and materials in locations prescribed
o (2) Exposure/Dosage = number, frequency, length initially programmed for the intervention to work
o (3) Quality of program delivery = manner in which person intervening delivers a program
o (4) Participant responsiveness = extent to which participants are engaged by and involved in the activities and content of the program
o (5) Program differentiation = is defined as “identifying unique features of different components or programs”, and identifying “which elements of…programmes are essential”, without which the programme will not have its intended effect (as cited in Carroll, 2007)
More fidelity, greater behavior change
The greater the number of modifications present, the greater the likelihood that key components linked to effectiveness are altered/removed; danger of Type III error
Breitenstein, 2010
ARGUMENTS FOR IMPLEMENTATION FIDELITY: Preservation of causal mechanism
“Using implementation fidelity information in the analysis of intervention effectiveness is important because fidelity outcomes (a) are related to the internal validity of an intervention study, (b) allow increased confidence in attributing improvements to the intervention, and (c) may increase statistical power by controlling for error associated with diminished implementation quality”
Carroll, 2007
ARGUMENTS FOR IMPLEMENTATION FIDELITY: Preservation of causal mechanism
“It is only by making an appropriate evaluation of the fidelity with which an intervention has been implemented that a viable assessment can be made of its contribution to outcomes, i.e., its effect on performance”
EXAMPLE: Welsh National Exercise Referral Scheme (NERS), Moore et al., 2013
In the study of the Welsh National Exercise Referral Scheme (NERS), Moore et al. (2013) provided evidence on how the poor implementation of the exercise referral intervention resulted in ambiguous process evaluation outcomes
The NERS aimed to increase the physical and mental health of the at-risk population by improving their involvement in a group physical activity
HOWEVER, the NERS program developers overlooked the importance of maintaining a reliable monitoring system throughout the implementation [risk to implementation fidelity]
The absence of the monitoring system resulted in the limited delivery of the motivational interviews (MI) and one-to-one consultations between the patients and the exercise professionals
Thus, the lack of implementation fidelity negatively affected the final process evaluation that, in turn, could not adequately assess the effectiveness of the whole intervention
Durlak, 2008
ARGUMENTS FOR IMPLEMENTATION FIDELITY: Essential for replicability
“Research indicates that the diffusion of effective interventions typically yields diminishing returns as the process enfolds”
Morrison et al., 2009
ARGUMENTS FOR CULTURAL ADAPTATION
“Whereas the ‘gold standard’ for a replication had been strict fidelity to the content and procedures of the original project, we now understand that slavish [uncompromising] fidelity may result in an intervention that is faithful to the form, but not the essence, of the original”
Backer, 2001
ARGUMENTS AGAINST IMPLEMENTATION FIDELITY: The study of implementation process has been limited to date
Durlak, 1997
ARGUMENTS AGAINST IMPLEMENTATION FIDELITY: The study of implementation process has been limited to date (Backer, 2001)
Noted that less than 5 percent of more than 1200 published prevention studies provide data on program implementation
Durlak and Wells, 1998
ARGUMENTS AGAINST IMPLEMENTATION FIDELITY: The study of implementation process has been limited to date (Backer, 2001)
Meta-analysis of prevention programs found that 68.5 percent were described too broadly to be replicated and few included measurement of fidelity
Gresham, Gansle, Noell, Cohen, and Rosenbaum, 1993
ARGUMENTS AGAINST IMPLEMENTATION FIDELITY: The study of implementation process has been limited to date (Backer, 2001)
Found that only 14.9 percent of reviewed school-based prevention programs measured implementation integrity
Dane and Schneider, 1998
ARGUMENTS AGAINST IMPLEMENTATION FIDELITY: The study of implementation process has been limited to date (Backer, 2001)
Assert that “promoting” integrity (e.g., use of a manual, providing training, etc.) is not the same as “verifying” integrity (gathering research data that these interventions were effective in high-quality implementation)
Baumann et al., 2014
ARGUMENTS AGAINST CULTURAL ADAPTATION: Costly, time-consuming
Parents in the intervention group of Baumann et al.’s Parent Management Training Oregon (PMTO) study in Mexico were engaged in activities such as training and coaching by local PMTO mentors
HOWEVER, it took 18 months to train the local PMTO coaches at the professional PMTO and the fidelity implementation (the FIMP) training before the intervention
Shaffer et al., 1990
Demonstrating the need for developers of widely-delivered programmes to be cognizant of any potentially significant variations within the intended receiving population, numerous impact assessments of suicide-prevention curricula implemented in school settings indicate that among students exposed to a given programme, those who have disclosed a previous suicide attempt prove less likely to recommend the programme to their peers and more likely to experience feelings of distress as a result of the programme content
Overholser, Hemstreet, Spirito, and Vyse, 1989
Demonstrating the need for developers of widely-delivered programmes to be cognizant of any potentially significant variations within the intended receiving population, male students in general display increased levels of hopelessness and worsened ‘maladaptive coping responses’ after exposure to suicide awareness curriculum
Cho and Salmon, 2007; Guttman and Zimmerman, 2000
Studies suggest that the use of mass media campaigns as a vehicle for disseminating health-related messages may result in group-specific iatrogenic outcomes, negatively affecting the well-being of a targeted population by inducing states of guilt and psychological discomfort, potentially most severely felt among individuals who lack the resources necessary to bring about an advocated change
Rose et al., 2002
Adverse psychological effects have also been identified in relation to the one-time ‘debriefing’ sessions frequently provided in the wake of exposure to a traumatic incident:
o Not only does the available evidence on the subject of single-session psychological debriefings fail to substantiate the theorized reduction in patient risk of developing post-traumatic stress disorder (PTSD), but some trials find increased incidence of PTSD and depression among participants who received the debriefing
Chalmers, 2003
From a pragmatic perspective, conducting social interventions without having first consulted systematic reviews and evaluated potential for harm seems highly unlikely to prove an efficient employment of (undoubtedly limited) resources
Rather, as Iain Chalmers writes, such a failure carries “human and financial costs”
Rothman, Greenland, and Lash, 2008
Ethical obligations toward control groups include the concept of equipoise, which entails ensuring that current best practice deems all treatments included in a trial to be equally ‘acceptable’
Bonell et al., 2015
Workable ‘dark logic’ models should be constructed before intervening in order to evaluate the potential iatrogenic effects of a social intervention
o Ethical imperative to consider the potential for harm in advance (a priori) and to design our evaluations to appropriately assess for any such iatrogenic outcomes or mechanisms
Bien et al., 1993
For alcohol misuse, the following elements have been identified as particularly important, forming the acronym “FRAMES”:
o Feedback on the risk for alcohol problems
o Responsibility: where the individual with alcohol misuse is responsible for change
o Advice: about reduction or explicit direction to change
o Menu: providing a variety of strategies for change
o Empathy, with a warm, reflective, empathic and understanding approach
o Self-efficacy of the misusing person in making a change
FEEDBACK on the risk for alcohol problems
o “Most brief interventions have provided clients with feedback, in some form, of assessment results. Evaluation has often, but not always, been extensive, occupying an average of 2-3 hours…Simple participation in this amount of evaluation may produce an impact through the inherent feedback involved (e.g. Elvy et al., 1988; Heather et al., 1986). Some brief interventions have focused explicitly on feedback of impairment as a tool for instigating change (e.g. Kristenson et al., 1983; Miller and Sovereign, 1989; Persson and Magnusson, 1989)”
Emphasis on personal RESPONSIBILITY for change
o “Brief interventions have also commonly advised patients that their drinking is their own responsibility and choice. An implicit or explicit message is that ‘No one can make you change, or decide for you. What you do about your drinking is up to you.’ Perceived personal control has been recognized as an element of motivation for behavior change and maintenance (Deci, 1975; Miller, 1985)”
Clear ADVICE to change
o “Nevertheless, all of the interventions described in these studies have contained explicit verbal or written advice to reduce or stop drinking. Advice has, in fact, sometimes been described as the essence of the brief intervention (Edwards et al., 1977; Orford and Edwards, 1977)”
A MENU of alternative change options
o “Self-help resources have typically described not one, but an array of alternative strategies for reducing drinking (Heather et al., 1989; Miller and Mofioz, 1982; Robertson and Heather, 1986). Brief interventions, as described in these studies, have seldom prescribed a single approach, but have advised either a general goal or a range of options. Presumably, this increases the likelihood that an individual will find an approach appropriate and acceptable to his or her own situation”
Therapeutic EMPATHY as a counseling style
o “Whenever counselor style has been described in reports of effective brief interventions, the emphasis has been on a warm, reflective, empathic, and understanding approach. No reports of effective brief counseling have resembled the directive, aggressive, authoritarian, or coercive elements that are sometimes associated with alcohol/drug abuse counseling (Miller, 1985; Miller and RoUnick, 1991), though some have called their interventions ‘confrontational’ (e.g. Chapman and Huygens, 1988; Sannibale, 1988)”
Enhancement of client SELF-EFFICACY or optimism
o “Finally, it is common in brief intervention to encourage clients’ self-efficacy for change, rather than emphasizing helplessness or powerlessness. Optimism regarding the possibility of change has often been embedded in effective motivational counseling”
+ ongoing follow-up
o “In addition to these six FRAMES elements, brief intervention studies have also included repeated follow-up visits. This was a substantial difference between intervention and control groups in some studies (e.g. Kristenson et al., 1983; Wallace et al., 1988). The occurrence of first follow-up, in fact, has been following by a reduction in drinking among controls in at least two studies (Elvy et al., 1988; Heather et al., 1987), and sustained follow-up has more generally been recognized as a factor favoring change and maintenance. Nevertheless, many of the above trials document the occurrence of marked behavior change immediately following the brief intervention, without the benefit of repeated follow-ups”
“Brief interventions in health care settings have been tested against untreated controls in trials conducted in 14 nations. Among a dozen studies of procedures designed to increase referral or retention in specialist services, all but one found significant effects of a brief intervention. Significant reductions in alcohol use and/or related problems have been reported for brief drinking-focused interventions (v. no counseling) in seven of eight randomized trials in healthcare settings. This places brief counseling among the most strongly supported intervention modalities for alcohol problems, and certainly as the most cost-effective, based on currently published clinical trials (Holder et al., 1991)”
“Perhaps the best promise of brief motivational counseling…is as a low-cost intervention that can be applied to large populations within the confines of ongoing service delivery systems.
o The results from this substantial body of clinical trials are remarkably consistent across cultures: brief intervention yields outcomes significantly better than no treatment, and often comparable to those of more extensive treatment.
o Most problem drinkers never come into contact with specialist treatment services, where the focus is often on alcoholism or alcohol dependence, but do frequent their health, social, and employment service systems.”
“As lay and professional conceptions shift toward a public health ‘alcohol problems’ perspective (Cahalan, 1987; Institute of Medicine, 1990; Moore and Gerstein, 1981; Royal College of General Practitioners, 1986), brief interventions offer a suitable and encouragingly effective option to be implemented as part of routine care”
Moyer et al., 2002
A number of randomised controlled trials have shown that, in comparison with controls, hazardous and harmful drinkers receiving 5-10 minutes of brief structured advice plus a self-help booklet from primary health care workers will reduce alcohol consumption by an average of 25%
WHO Collaborative Project on Detection and Management of Alcohol-related Problems in Primary Health Care
The WHO Collaborative Project has been concerned with developing, testing and implementing screening and brief alcohol intervention in primary health care settings
Overall, it has been estimated that around 20% of patients identified as hazardous or harmful drinkers who receive a brief intervention will reduce their alcohol consumption
Because research has shown that brief interventions are low in cost and have proven to be effective across the spectrum of alcohol problems, health workers and policy-makers have increasingly focused on them as tools to fill the gap between the primary prevention efforts and more intensive treatment for persons with serious alcohol use disorders
Phases of the WHO project:
o [Phase I]: a reliable and valid screening instrument for detecting hazardous and harmful drinkers in primary health care (PHC) settings was developed (i.e., the AUDIT questionnaire)
o [Phase II]: a clinical trial of screening and brief intervention in PHC was carried out; the effectiveness of brief interventions in primary health care was demonstrated in a cross-national randomised controlled trial
o [Phase III]: the current practices and perceptions of general medical practitioners (GPs) were assessed, in-depth telephone interviews with GPs and personal interviews with key informants were conducted, and methods for encouraging the uptake and utilization of a screening and brief intervention practice by GPs were evaluated in a controlled trial
o [Phase IV]: concerned with the development and application of strategies for the widespread, routine and enduring implementation of screening and brief alcohol intervention in primary health care throughout participating countries
Problems/limitations of brief interventions for alcohol (mis)use:
o (1) Attitudes of primary health care professionals: One of the central problems was in persuading PHC professionals to become engaged in the research
o (2) Funding: Another serious problem in many participating countries was an inability to obtain necessary funding from national bodies for aspects of the Phase IV research
o (3) Lack of government support: Lack of support from local, regional, and national governmental authorities for including alcohol screening and brief interventions (SBI) in health promotion campaigns and strategies, and in plans for the regulation and reimbursement of PHC activity
o (4) Economic evaluation: Limited evidence available to attest to the cost-offsets of SBI in contexts outside the United States, where it has been demonstrated that the costs of delivering SBI in PHC are more than offset by reductions in costs of the future use of health services (Fleming et al., 2000)
Achievements of brief interventions:
o (1) Customisation: All participating countries succeeded in making adaptations to materials or procedures involved in SBI implementation to suit the requirements of the local health system and sociocultural setting
o (2) Reframing: Though it did not prove possible to devise a valid and reliable measuring instrument to record the cognitive and attitudinal changes that the reframing process aimed for, the notion of hazardous or “risky” drinking and its relevance to public health was nonetheless introduced to a range of PHC professionals and other stakeholders through formal training programmes and other avenues
o (3) Lead organization and strategic alliance: Establishment in each participating country of a lead organization for the country-wide implementation of SBI, with endorsements from a range of organizations and individuals that are highly relevant to the aims of the study in each country
o (4) Demonstration project: “Demonstration projects” (Higgins-Biddle and Babor, 1996), some controlled in various ways and some simply before-after designs, made some contribution to demonstrating how widespread implementation can be achieved in the circumstances of the country in question
Fleming et al., 2000
Problems/limitations of brief interventions for alcohol (mis)use:
o Economic evaluation: Limited evidence available to attest to the cost-offsets of SBI in contexts outside the United States, where it has been demonstrated that the costs of delivering SBI in PHC are more than offset by reductions in costs of the future use of health services
Higgins-Biddle and Babor, 1996
Achievements of brief interventions:
o Demonstration project: “Demonstration projects”, some controlled in various ways and some simply before-after designs, made some contribution to demonstrating how widespread implementation can be achieved in the circumstances of the country in question
Cahalan, 1987; Institute of Medicine, 1990; Moore and Gerstein, 1981; Royal College of General Practitioners, 1986
Shift of lay and professional conceptions toward a public health ‘alcohol problems’ perspective
Alcohol Use Disorders Identification Test (AUDIT)
Validated tool developed by the World Health Organization (WHO) to assess a person’s drinking risk level (in the form of a ten-question self-report survey)
Heather, 1994, 1995
Brief interventions for excessive drinking should not be referred to as an homogenous entity, but as a family of interventions varying in length, structure, targets of intervention, personnel responsible for their delivery, media of communication and several other ways, including their underpinning theory and intervention philosophy
Brief interventions, therefore, can be viewed as a set of principles regarding interventions which are different from, but not in conflict with, the principles underlying conventional treatment
Holland, 1986
Addresses the issue of the counterfactual in what he terms the Fundamental Problem of Causal Inference, which states that “[i]t is impossible to observe the value of Yt(u) [treatment] and Yc(u) [control] on the same unit and, therefore, it is impossible to observe the effect of t on u”
Godwin et al., 2003; Steckler and McLeroy, 2008
The social intervention literature proposes a distinction between ‘efficacy’ and ‘effectiveness’ trials, similarly known as ‘explanatory’ and ‘pragmatic’ trials, respectively
o The former categorization refers to those trials testing whether a causal effect can be found in an intervention conducted under ideal circumstances, while the latter denotes trials that work to more fully incorporate the nuances of ‘real-life’ settings, thereby striking a balance between considerations of internal and external validity that more closely approximates equivalence
As originally conceived, efficacy/explanatory trials—those most concerned with meeting the demands of internal validity—necessarily precede effectiveness/pragmatic trials, indicative of a general belief shared among many social researchers that causal relationships must be verified and well-understood before they can be generalized (Flay, 1986):
o If we fail to fully comprehend the ‘why’ behind a causal relationship, or incorrectly identify the cause associated with an observed effect, then we find ourselves unable to validly extrapolate from our results to other times, populations, settings, and measures
Shadish, Cook, and Campbell, 2002
Definition of internal validity:
o Three conditions must be satisfied in order to draw a valid causal inference: the intervention (i.e., “cause”) must temporally precede the outcome(s); the intervention and outcome(s) must covary; and all alternative explanations for this covariance must be proven implausible
Hajizadeh et al., 2016
Proper use of prenatal care cannot be achieved merely by establishing healthcare centres, and instead requires addressing a range of socioeconomic barriers
Kita et al., 2015
Identifies the most common barriers to accessing prenatal care among immigrant women as financial constraints, lack of insurance coverage, transportation problems and linguistic and cultural differences
Furthermore, the stress of adjusting to a new country has also been found to contribute to underutilization of prenatal care services among women who have recently immigrated
Machado et al., 2009
Immigrant children are at increased risk for health problems such as respiratory and ear infections, gastroenteritis, skin infections, dental problems, anemia and infectious diseases such as TB
Additionally, during the immigration process, infants and children tend to be more exposed to intentional and unintentional injuries and mental health problems
Peters et al., 2008
Four most commonly identified barriers to health care: (1) availability, (2) affordability, (3) accessibility and (4) acceptability
Edgerley et al., 2007
Women who utilized a mobile health clinic in an underserved community in California initiated prenatal care approximately three weeks sooner than women who used standard community health services
o There was no difference between mobile and standard services in quality of care received
Lagarde et al., 2007
Income support programmes with conditionality attached on utilization of health services can have a positive impact on children’s health outcomes, nutritional status, and uptake of preventive services by pregnant women and children
Diversity Initiative in Ukraine (International Organization for Migration [IOM], 2017; Miller, 2018)
The Diversity Initiative in Ukraine is a broad-based network of strategies designed to raise awareness and promote diversity amongst the general public in addition to supporting the Ukrainian government in assisting populations that suffer from xenophobia and hate crimes
The Initiative was implemented in response to racially motivated attacks in Ukraine in 2006 and has been successful in reducing the prevalence of such incidents
Rade et al., 2018
Cultural illiteracy embedded within health services impedes sexual health help-seeking behavior among migrants from sub-Saharan Africa and South East Asia living in high income countries
World Health Organization (WHO, 2012) definition of intimate partner violence (IPV)
IPV = “any behaviour within an intimate relationship that causes physical, psychological or sexual harm to those in the relationship”
Kimmel, 2002
The cumulative body of evidence overwhelmingly attests to a continued ‘gender asymmetry’ in domestic violence (the broader umbrella under which IPV falls), with men more likely than women to commit acts of violence against their partner, and women at significantly greater risk of experiencing abuse of a ‘control-motivated’ nature, as well as violence that results in more severely injurious health outcomes
Ellsberg et al., 2015
There exist a variety of interventions intended to prevent and/or reduce the prevalence of IPV, including group training initiatives, efforts to mobilize community stakeholders, economic empowerment programmes, educational campaigns, and advocacy interventions
Rivas et al., 2015
Systematic review of advocacy interventions delivered to women in the wake of intimate partner abuse (IPV)
Define advocacy as consisting of the following four core activities: “providing legal, housing, and financial advice; facilitating access to and use of community resources such as refuges or shelters, emergency housing, and psychological interventions; giving safety planning advice; and providing ongoing support and informal counselling”
Conclude that, though intensive advocacy interventions may hold some promise for improving the short-term outcomes of abused women residing in domestic violence shelters, and may reduce incidence of physical abuse in the one-to-two year period following delivery of an intervention, there lacks equally compelling evidence for the effectiveness of brief interventions, and insufficient evidence to claim that intensive advocacy interventions result in better mental health outcomes for abused women, or reduced levels of emotional, sexual, or overall abuse
Balas and Boren, 2000
The historically slow and arduous path from research to practice—with some studies finding a delay averaging 17 years between the initial publication of evidence-based findings and their eventual incorporation into clinical practice—clearly elucidates the need for a better understanding within the evidence-based practice movement of the guiding priorities of practitioners and policymakers
Brownson and Jones, 2009
“In practice settings, effective dissemination of an evidence-based program often calls for time-efficient approaches, ongoing training, and a high organizational value on research-informed practice…In addition, practitioners often seek: a strong connection to community needs, realistic and economically-feasible intervention options, and leadership capacity to translate research to policy”
Chris Bonell: Realist RCTs
Acknowledge limitations of (some) RCTs and insights from realist evaluation
Aim to take realist approach, but within RCT
o Use theory which goes beyond logic models to describe mechanisms and contextual contingencies
o Refine theory using embedded qualitative research
o Test hypotheses concerning what works for whom and how using moderator and mediator statistical analyses
Possible to pursue realist approaches within RCT
Aim is to refine theory > accredit as ‘effective’
Plenty of variation in intervention and context enabling assessment of how mechanisms vary with context
Key limitations:
o Not having quantitative measures of all phenomena apparent from qualitative research
o Lacking power for moderator analyses
–> But these limitations not particular to RCTs
Medical Research Council (MRC) guidance: Process evaluations (Moore et al., 2014)
“a study which aims to understand the functioning of an intervention, by examining implementation, mechanisms of impact, and contextual factors”
“complementary to, but not a substitute for, high quality outcomes evaluation”
Implementation: process through which interventions are delivered, and what is delivered in practice. Key dimensions:
- Implementation process: structures, resources and mechanisms through which delivery is achieved
- Fidelity: consistency of what is implemented with the planned intervention
- Adaptations: alterations to an intervention to achieve better contextual fit
- Dose: how much intervention is delivered
- Reach: extent to which a target audience comes into contact with the intervention
Mechanisms of impact: intermediate mechanisms through which intervention activities produce intended (or unintended) effects). May include:
- Participant responses: how participants interact with a complex intervention
- Mediators: intermediate processes which explain subsequent changes in outcomes
- Unintended pathways and consequences
Context: factors external to the intervention which may influence its implementation / whether its mechanisms of impact act as intended. May include:
1. Contextual moderators, which shape, and may be shaped by, implementation, intervention mechanisms, and outcomes
Complex intervention: “an intervention comprising multiple components which interact to produce change. Complexity may also relate to the difficulty of behaviours targeted by interventions, the number of organisational levels targeted, or the range of outcomes”
