Dell'Acqua, F., McFowland III, E., Mollick, E. R., Lifshitz-Assaf, H., Kellogg, K., Rajendran, S., ... & Lakhani, K. R. (2023). Navigating the jagged technological frontier

Question 1

• Large Language Models (LLMs):

Answer 1

LLMs, like GPT-4, have dramatically influenced how
knowledge workers perform tasks. Their ability to overlap human-like capabilities in writing,
analytics, and creativity has pushed the boundaries of knowledge work automation.

Question 2

Jagged Technological Frontier

Answer 2

AI capabilities create a jagged frontier, excelling in some areas
but underperforming or misleading in others. Tasks that seem similar in difficulty may yield
starkly different outcomes when AI is involved.

Question 3

Experimental Design:
- Study Context:

Answer 3

Conducted with 758 consultants from the Boston Consulting Group (BCG),
representing 7% of its global consultant workforce.

Question 4

• Three Experimental Conditions:

Answer 4

• No AI access (Control Group).
• AI access (GPT-4).
• AI access with prompt engineering training (GPT + Overview).

Question 5

• Task Nature

Answer 5

Consultants were assigned 18 realistic, complex consulting tasks spanning
creativity, analytics, writing, and persuasion. Additionally, a task outside AI’s capabilities was
included for comparison.

Question 6

Findings on Performance (Inside the Frontier of AI Capabilities)
- Quality Improvement:

Answer 6

• AI significantly boosted the quality of task outputs by over 40% on average compared to
  the control group.
• Consultants with AI completed 12.2% more tasks and worked 25.1% faster.
• Even lower-performing consultants benefited, with productivity improvements of up to
  43%, compared to 17% for higher-performing peers.

Question 7

• Differentiating AI Use Cases:

Answer 7

• Consultants using GPT + Overview outperformed those with GPT-4 alone due to better
  understanding and application of AI-assisted tools.
• Performance benefits were consistent across demographics, suggesting broad
  applicability.

Question 8

• Collaboration Patterns:

Answer 8

Centaurs, Cyborgs

Question 9

• Centaurs:

Answer 9

Strategically divide tasks between human expertise and AI strengths

Question 10

• Cyborgs:

Answer 10

Fully integrate AI into workflows, alternating and interweaving efforts.

Question 11

Challenges with AI (Outside the Frontier of AI Capabilities)
- Accuracy Issues:

Answer 11

• Tasks outside the frontier showed reduced performance for AI-augmented consultants.
• AI users were 19% less likely to provide correct solutions compared to non-AI users.
• Reliance on AI inappropriately for complex analytical tasks led to errors.

Question 12

• Time Efficiency vs. Accuracy:

Answer 12

• While AI reduced time spent on tasks by up to 30%, the quality of outcomes (correctness)
  often suffered.