L13 - MST evaluation Flashcards
What is the systematic approach to evaluation (Anderson et al, 2002)??
Why -> how -> who -> what -> when -> criteria of effectiveness -> why
Cyclical manner
Why should we evaluate MST? (Chelmsky, 1997)
- render judgement -> have intervention goals been met?
- facilitatate improvement -> what are main strengths /weaknesses
- generate knowledge -> why did it work?
Discuss ‘why’ element of Anderson et al (2002) cyclical diagram?
To be accountable to stakeholders
- client (athlete)
- secondary client (coach, NGB)
- themselves
- the sport psychology profession
Discuss positives and negatives of the experimental approach (MST)?
Positives - high internal validity
Negatives - lacks ecological validity, impractical inappropriate to withhold services
Discuss the positives and negatives of the case study approach (MST)??
Positives - holistic evaluation picture, no control group, accommodate practice setting / individual needs
Negatives - weak internal validity, need to be rigorous and systematic
When would one choose either the experimental or case study approach??
Experimental - choose if priority is to ‘render judgements’ cause and effect
Case study - choose if priority is to ‘facilitate improvement and generate knowledge’
Who will evaluate?? (Anderson et al, 2002 cyclical model)
External - achieves greater objectivity, may fail to capture subtleties of context, little value to actual practice, lack of ownership
Internal - by those who delivered the programme, greater relevance, gives a sense of ownership, changes more likely to occur following evaluation
Who should be included in evaluation of MST (Anderson et al 2002)
Managers, practitioner, parents, coach and participants
‘Wherever possible, practitioners should aim to conduct a comprehensive evaluation by collecting information from multiple sources’
What should be evaluated in MST (Anderson et al, 2002)
- quality of support
- athlete responses to support
- psychological skills and well being
- performance
Discuss ‘quality of support’ in more detail?
- participants fill out sport psychology consultant evaluation form (CEE; Partington and Orlick, 1987)
Social validation - helps to ensure practitioners do the best job that they can in helping consumers of their service function to the best of their ability
-increases in rugby performance as a result of goal-setting intervention were perceived as effective by players and the changes in performance were viewed as useful by the team (Mellalieu et al,2006)
Discuss social validation (component of quality of support) in more detail?
Social significance of the goal - what do the participants think about the goals of the intervention?
Social appropriateness of procedures - what do they think about the procedures that were applied?
Social importantness of intervention effects - what do they think about the results produced by these procedures?
What are methods of measuring social validation?
Questionnaires (open- ended questions, rating scales)
Semi-structured interviews
Discuss Psychological skills and well being? (Evaluation of MST)
Psychological skills - e.g., build/maintain confidence/self efficacy, regulate anxiety/intensity, control emotions
Well-being - e.g., positive/negative affect, enjoyment, self-esteem
Discuss athlete responses to support (evaluation of MST)?
- athletes responses to sport psychology services can influence the overall effectiveness of the support (Vealey, 1994)
What to evaluate? - changes in knowledge, changes in attitudes, adherence and use
Discuss performance in relation to evaluative properties of MST?
- objective - ranking, win/loss ratio, handicap
- subjective - satisfaction with performance, encourages athlete to record, reflect and learn from performances
A combined approach to provide a more complete picture (Holder, 1997)
