Evidence-based Management (Pfeffer & Sutton)

Question 1

What is evidence-based management?

Answer 1

The conscientious, explicit, and judicious use of current best evidence to solve organisational problems.

Question 2

What is the premise of evidence-based management?

Answer 2

Using better, deeper logic & employing facts to the extent possible, permits leaders to do their jobs more effectively.

Question 3

What is key in evidence-based management?

Answer 3

• Data
• Facts
• Evidence

Question 4

What is left out of evidence-based management?

Answer 4

• Formal authority
• Reputation (of manager)
• Intuition

Question 5

What are the six substitutes for best evidence?

Answer 5

Some decision-makers rely too heavily/solely on:
- Obsolete knowledge
- Personal Experience
- Specialist Skills
- Dogma & belief
- Casual Benchmarking
- Hype & Marketing

Question 6

Why is casual benchmarking not best evidence?

Answer 6

Firms tend to copy the other firm’s practice, without thinking about the logic the competitive advantage (essence of CA - culture, management, philosophy). All firms have a different strategy, informed by their internal structure, so there is a lot more analysis (or questioning) that should go into a firm’s decision on whether to adopt (& imitate) a practice. (Remember: copying other companies leads to, at best, a perfect imitation - you can’t beat the top performer in this way - so avoid mindless mimicry of top performers).

Question 7

So can benchmarking produce evidence?

Answer 7

Yes, but first, managers should ask:
(1) Do sound logic & evidence indicate that the benchmarking target’s success is attributable to the practice we seek to emulate?

(2) Are the conditions at our company - strategy, business model, workforce - similar enough to those at the benchmarked company to make the learning useful (seek to learn from relevant others)?

(3) Why does a given practice enhance performance? And what is the logic that links it to bottom line results? (Particularly important if firm chooses to imitate).

(4) What are the downsides of implementing the practice even if its is a good idea overall? (Find ways of mitigating the downsides, maybe even solutions that your benchmarking target uses that you aren’t seeing).

Question 8

Example: Cisco

Answer 8

Cisco’s success with mergers is hard to replicate. The firm has focused on developing this practice as a capability, it’s history, learnings, social network, cannot be easily substituted.

Question 9

Example of failed imitation

Answer 9

United Airlines attempted to copy Southwest with it’s “fighter airline”, Shuttle by United. They failed to remain in the low-cost carrier industry.

Question 10

Example of a practice as philosophy

Answer 10

Total Quality Management (TQM) at Toyota.

Question 11

According to Pfeffer & Sutton, how do you know you are making a decision based on current best evidence?

Answer 11

First, you turn the current situation into an answerable question.

Second, you consider the strengths/ flaws of the research you have gathered, critically appraising the research method, data, and survey.

Third, you gather more evidence, also weighing the negative against the positive.

Finally, you make a decision, based on this evidence-based approach.

Question 12

Example: Forced Ranking System for Talent Management

Answer 12

(1) Question: Should the firm adopt forced ranking of its employees?

(2) Data: Is there any data supporting forced ranking?(i) The War for Talent -
research method was flawed - the study violates a fundamental condition of causality - the proposed cause needs to occur before the proposed effect.

(3) Gather more evidence:
(i) Performance improves with team continuity & time in position (evidence from different sectors).
(ii) In work that requires cooperation, performance suffers when there is a big spread between the worst-and-best-paid people (Haas School of Business Study).
( iii) Novations Group Survey, over 200 Human Resource professionals, from companies with more than 2,500 employees (half of the companies used forced-ranking).

Question 13

What did the evidence on joint experience reveal (3) (i)?

Answer 13

Performance success is enhanced with communication, mutual understanding, & the ability to work together over a substantial period of time (company work involves cooperation, coordination & information sharing).

Question 14

What did the Novations Group Survey reveal about forced ranking?

Answer 14

Dispersed pay or forced ranking results in:
- lower productivity
- inequity (some persons missing out when they have a need)
- skepticism
- decreased employee engagement
- reduced collaboration - damage to morale
- mistrust in leadership

Question 15

What did Wharton’s John Paul MacDuffie’s 15 years of quantitative studies reveal?

Answer 15

John MacDuffie found that automobile firms with lean or flexible production systems - with emphasis on teams, training, & job rotation, & their de-emphasis on status differences among employees build higher-quality cars at a lower cost.

Question 16

Six Standards for producing, evaluating, selling, & applying business knowledge

Answer 16

• stop treating old ideas as if they were brand new (acknowledge key sources & build & blend on what’s come before).
• be suspicious of “breakthrough” ideas & studies (avoid business gurus who claim a brand-new-cure-all).
• celebrate & develop collective brilliance (recognise the value of teams and communities of researchers - the actions and commitment of many people to these ideas are crucial for accomplishing organisational change)
• emphasise drawbacks as well as virtues (all management practices & programs have strong & weak points - knowing the hazards helps managers prepare - not abandon a program or practice when setbacks occur)
• use success (& failure) stories to illustrate sound practices, but not in place of a valid research method (rely on management research, limit reliance on recollections by parties involved in a project)
• adopt a neutral stance toward ideologies & theories (establish clarity & consensus on what problem needs to be solved & what constitutes evidence of efficacy - be open to using & learning from new evidence).

Question 17

So what can managers do to really apply an evidence-based practice approach when making decisions for the company?

Answer 17

Always ask yourself, will this practice work for our company? Then set out on a process of determining why or why not by gathering data & evidence. This evidence must be evaluated, so that managers can make an informed decision.

Question 18

How can you implement evidence-based practice in organisations?

Answer 18

Have an attitude of wisdom by demanding evidence, examining logic (behind practices) and aim to treat the organisation as an unfinished prototype.

Question 19

What does demanding evidence look like?

Answer 19

Ask for evidence of efficacy every time a change is proposed because we do not want to be (mis) guided by half-truths.

Question 20

Benefit of demanding evidence

Answer 20

• Helps the organisation develop its own evidence base.

Question 21

Example: Davita

Answer 21

• Culture: commitment to the quality of patient care.

Action:
(I) Begin meeting with data by getting employees to produce 8 page reports monthly on quality index metrics (i.e. quality of care).
( ii) Highlight metrics that are missing - encourages the company to solve for gathering this missing information.

Question 22

What does examining logic look like?

Answer 22

Pay close attention to gaps in exposition, logic, or inference by asking yourself:
- Is this experimental research (scholarly evidence)? Or “non-experimental research? ( data from company records).

• Is it a sensible assumption? Leaders should tap into the collective wisdom of their teams to explore whether the underlying assumptions of a policy, practice, or intervention seem sensible.

Question 23

What does treating the organisation as an unfinished prototype look like?

Answer 23

Research about the company using its own data base & experience (conduct rigorous study of practices & gather qualitative & quantitative data).

Question 24

Example of rigorous study within organisations

Answer 24

• EBay: responded quickly to target market feedback & fixed “problems” on the fly.
• Yahoo: conducted 20 or 30 so experiments at one time. Little experiments can have big effects (i.e. moving the search box to the centre of the homepage produced more “click through” & brought in more advertising revenue a year).

(Include field experiments, make comparisons with “control” locations).

Question 25

What does embracing an attitude of wisdom look like?

Answer 25

• Come to realise that you don’t know everything (humility)
• Act on the best of one’s knowledge, while questioning what you know (decisiveness).
• Encourage inquiry & observation when rigorous evidence is lacking & you need to act quickly.

Question 26

Question: Every firm has its own strategy, so with benchmarking, you also want to apply a structured & logical approach to make an informed decision.

Answer 26

Strategy is one very important consideration, so if a firm uses benchmarking as a form of EVIDENCE, which would have more weight in decision-making?
Benchmarking or the scholarly evidence? in the article, forced ranking of employees worked for GM, but not for another automobile company.

My question is: If we determine that the benchmarking target’s practice could work well in the company, yet evidence tells us that the practice is not as beneficial for imporving employee engagement, for example - what should we base our decision on? Would you say strategic fit carries more weight, or research evidence is most important?