Story #2: VMI Measurement

Question 1

Interviewer Questions

Answer 1

1. Tell me about a time when you initiated work on a project that both impacted a majority of your team and had a lot of opposition. ←- THINK BIG
2. Tell me about a time when you had to make a bold and difficult decision. ←- THINK BIG
3. Tell me about a time when everyone else on your team gave up on something but you pushed the team towards delivering a result. ←- DELIVER RESULTS
4. Tell me about a time where you made a decision without having complete information. ←- ARE RIGHT A LOT
5. Tell me about a time you made a design decision where a lot of people had opposed you. Why did they oppose you? ←- ARE RIGHT A LOT

Question 2

How would I open VMI Measurement?

Answer 2

Let me tell you about a time when I helped Old Navy Inventory Planners better measure and improve their ability to predict how much inventory we should send for an item to a certain location at a point in time.

Question 3

What is the framework for telling a story?

Answer 3

Situation, Action, Task, Result

Question 4

What is the framework for Situation?

Answer 4

Context - Vision
User
Problem
System
Good Thing
Bad Thing
Summary

Question 5

Who is the user for VMI Measurement? What is their job all about?

Answer 5

Old Navy Planners job is to try to make future predictions about the customer. needs to predict how much of an item, such as a pair of jeans, the customer will want to purchase at a location (Old Navy 34th street), at a point in time (in September of 2022). Once a planner understands future customer demand, she can predict how much inventory we need to send to a location, at a point in time, so that we can satisfy customer demand with supply.

Question 6

What is the problem?

Answer 6

Old Navy Planners need to nderstand how accurate they are at making these predictions. They don’t have a good way of measuring their accuracy.

Question 7

Whats the bad thing that happens today for Planners

Answer 7

They are frustated, and feel lost. They don’t know if the levers they are pulling to meet demand with supply are effective or not.

Question 8

What do planner need? What do they use today for help

Answer 8

Planners need the a clear measure of how effective they are at making inventory predictions. They need clarity and guidance through data.
They need to talk to the rest of the organization about performance - for example, talk to a store manager about how effective they were last month and how they can improve.
Today they use a reording system to help make predictions - think Google Maps.

Question 9

What my hyptohesis for solving VMI measurement?

Answer 9

What if we equip Old Navy planners with the right measure –> right metrics to make improvement

Question 10

What are my actions?

Answer 10

1. What to measure - Brainstorm KPIs and prioritize the most effective
2. How to measure - Data setup, Data cleansing
3. Building a Story - Review details of measurement success with business
4. Propose Improvements - measurement simulation
5. Implement & Measure(Tradeoffs, tough decisions)

Question 11

STEP 1: What to measure?

Answer 11

1. Alignment on the Problem - I brought together stakeholders from data science, inventory management, and product management to brainstorm the right KPI or set of KPIs.
2. KPI Brainstorming - Held brainstorming exercises to identify first align on the goal -
   how much inventory did we supply for the customer / how much did the customer actually demand
   Objective - ask the right question
3. Prioritize the best KPI - Brainstorm KPIs and prioritize the most effective
   Service Level %
   Lost Sales
   Fill Rate %

Question 12

STEP 2: How to measure?

Answer 12

1. Gather Data:
   
   1. Customer Demand with LAG - Let’s gather historical data on customer demand. Predicted customer demand is a guess about the future of how much of something the customer will purchase.
      Predictions about customer demand are constantly changing with new information, so what version of customer demand do we need to use to measure fill rate %?
      We need to use the last prediction that was able to impact how many jeans we ordered. This is the lead time.
   2. Inventory Predictions with LAG - If you’re planning to have jeans in a store for the first week of August, the latest you can make a decision on how many jeans you need would be 10 weeks prior to the first week of August. If it takes 10 weeks to create a product and deliver it to a store, then the lead time is 10 weeks. This is the last time you impact an order, so this is the version of customer demand that we need to use.
      Source: Demand forecasting engine - snapshots in time

Inventory we provided - for the week we are measuring fill rate %, how many jeans did we have available for the customer?
This is a function of how many jeans we ordered 10 weeks out, given how many jeans were predicted to be in stock for the customer
Source: Enterprise data warehouse backup files, snapshots in time <– HOW MUCH INVENTORY DID WE ORDER, HOW MUCH WAS IN TRANSIT, HOW MUCH DID WE PROJECT TO HAVE
2. Item Eligibility:
1. Which items were planned during this period?
2. Can we use long living items?
3. Cleanse Data:
1. Short Shipped - Clease out instances in which the vendor short shipped.
2. Budget Cuts - user zero’d out a receipt because their budget couldn’t handle it.
4. Summrize the Data:
* Summarize the data to identify outliers and set up our story - dashboard?
* We summarized into a digestible dashboard but items such as Men’s denim, Men’s suits, etc.
* Dashboard is helpful for telling a story later on, but also for identifying data outliers

Question 13

STEP 3: Building the Story, Telling the Story

Answer 13

1. Building the Story:
   
   1. Conitinous alignment on:
      
      1. Objective
      2. KPIs - what is fill rate %
      3. Items we measured
      4. Complexities of measuring data
      5. Outcome we’re driving towards - better predictions, better customer experience.
2. Telling the Story:
   
   • We presented fill rate for Old Navy as of June - August 2021 to give us an impression of how well Old Navy can meet customer demand:
     
     • Old Navy fill rate was 49%
   • Outcome Driven: The customer experience is impacted today by our inability to meet customer demand.
   • KPI enabled: FIll Rate percentage is the data that helps us understand exactly how far we are away from meeting customer demand.

Question 14

EXTRA QUESTION: How to measure?

Summarize the data to identify outliers and set up our story - dashboard?

Answer 14

We summarized into a digestible dashboard but items such as Men’s denim, Men’s suits, etc.
Dashboard is helpful for telling a story later on, but also for identifying data outliers

Question 15

STEP 4: What recommendations for improvements? Where are we going next?

Answer 15

1. Propose Improvements: Where are we going next?
   
   1. Other Reordering System (At a total style level versus size):
      
      1. Differences:
         System 1 orders at a size level
         System 2 orders at an item level and uses a size profile to break down need by size
         System 2 a more user friendly design, that will enable users to be more reactive to changing supply chain strategies
         System 2 had a fill rate of 59%
   2. Simulation: Let’s simulate using another reordering system to see if we can achieve a better fill rate
      Used historical data to simulate system 1 versus system 2, with the same inputs
      Inputs:
      Safety stock weeks
      Demand forecast
      Orders in transit and on hand
      Visual presentation setting - order and on hand minimum
      Minimum order quantity
      
      1. Conclusion: Fill Rate was 59% for the second system.
2. Propose Implementation: Can we move to this order reording system that will deliver a higher fill rate %?

Question 16

STEP 4: What recommendations for improvements? ← What stakeholder issue occurred here? How did I react?

Answer 16

• The results of fill rate in the new system were not consistently higher in all departments.
• Departments such as Men’s Denim, Boys Knit Sweaters, and Women’s accessories had a lower fill rate %
• What did I do?
  
  1. Made sure our stakeholders were heard - I acknowledged that yes these 3 departments had a lower fill rate in the new system versus the old.
  2. What can we learn from the data at aggregate?
     
     1. Each impacted department was within 5% of the new systems Fill Rate % - I partnered with data science to communicate that:
        
        1. All other departments had a higher fill rate percentage.
        2. Data Science partnership to communicate the the overall potential profit margin benefit for all departments on the new system exceeded that of the old system.
        3. Despite 3 departments being lower, directionally we as a company achieve a higher benefit on the new system.
  3. Risk Management Plan for Go Live: Proposed that we go live and measure all departments fill rate % post go-live.
     
     1. We take special focus on the impacted departments and help the Planners propose receipt parameter adjustments post go live that might help the

Question 17

STEP 5: Implement & Measure(Tradeoffs, tough decisions)

Answer 17

1. Go Live: We migrated Old Navy, 300 items and 16 departments, to the new Reordering System.
2. Results: We tracked fill rate for 3 months post go-live and on average items are between a 55% and 62% fill rate on the new system, higher than the old system of 49% avg.
   
   1. Parameter Tuning:
      
      1. Impact departments could have their parameters tuned.
         
         1. Does Customer Demand fluctaute and do we need to add in additional safety stock?
         2. Do we need to set an inventory minimum by store that needs to be higher, again to meet fluctuations in customer demand?