Chapter 10: Measurement and evaluation

Question 1

Why is evaluating important?

Answer 1

• Central: organization objectives
• Effectiveness of programmes

Question 2

What is formative research?

Answer 2

Monitoring before & during the program

Question 3

What is summative research

Answer 3

Assessment of program impact

Question 4

What is SMARRTT?

Answer 4

Specific
Measurable
Achievable
Realistic
Relevant
Targeted
Timed

Question 5

What is Objective-based model (Tyler, 1942)

Answer 5

Define objectives prior to evaluation

Question 6

Objectives can be?

Answer 6

• Cognitive, affective, conative

Question 7

At what levels can objectives be set?

Answer 7

Output
Outcome
Outgrowth
Outflow

Question 8

How have evaluation models evolved?

Answer 8

• the number of stages within the model increased
• variations in terminology of each stage emerged
• Frameworks adopted a more inter-disciplinary approach drawing form evaluation literature
• Growing emphasis on the importance of evaluation PR toward achieving organizational goals

Question 9

What are the three evaluation models?

Answer 9

• Preparation, implementation, and impact model
• Communication controlling model
• Integrated model of evaluation for strategic communication

Question 10

How works the preparation, implementation and impact model?

Answer 10

• Preparation assesses info that is at the basis of the programme
• Implementation assesses the tactics applied to the programme
• Impact asses the realization of goals (changes in knowledge, attitude, behavior etc)

Question 11

What are the 4 stages of the communication controlling model (2011)?

Answer 11

1. Inputs
   resources of personnel
2. Output
   Internal =efficiency within the organization
   external = content and coverage of messages
3. Outcome
   Direct = changes in stakeholder perception + knowledge
   Indirect = Changes in stakeholders’ attitude and behavior
4. Outflow
   How well communication managed to achieve the targets

Question 12

What are the 5 levels of evaluation in the ‘integrated model of evaluation’?

Answer 12

1. Inputs:
   Formative research
2. Activities:
   Design and media relations
3. Outputs:
   Publications, content, events
4. Outcomes:
   Awareness, attitude, behavior
5. Impact:
   Societal and organizational results. Can be bi-directional!

Question 13

What are the barriers to evaluation practice?

Answer 13

• Budget, time
• PR practitioners lack knowledge of evaluation
• Management hinders a scientific approach
• Attitude of PR staff themselves
• Unhappy with the available tools

Question 14

Three specific areas that often generate debate?

Answer 14

• The advertising value equivalency debate
• Return on investment
• Online evaluation

Question 15

Explain “the advertising value equivalence debate”:

Answer 15

What would the campaign have costed if the equivalent was spent on advertising?
- Starting point = advertising rate
- Multiply the length of a column of text, or air time by the AV rate

Heavily criticized
- News stories unequal advertisement
- Value of keeping issue OUT of the news
- PR story often vary. Advertisement appears repeatedly
- Weak basis for the PR value multiplied added to the calculation

Question 16

Explain “return on investment”:

Answer 16

• Focus on financial returns on costs
• Monetary value of PR output often problematic
• Alternative: Social Return on investment
  o Value created by building social capital and managing key relationships

Question 17

Explain “online evaluation”:

Answer 17

• Monitor and evaluate communication on Social Media
• Focus more on output measurement: click and likes (‘vanity metrics’)
• Engagement: clicks/likes versus comments

Question 18

Forrester (2007): four levels:

Answer 18

1. Involvement (visits, clicks, likes, shares)
2. Interaction (comments, Purchasing)
3. Intimacy (sentiment analysis)
4. Influence (recommendations)