Story #5 Assortment Optimization

Question 1

How do I open Assortment Optimization?

Answer 1

Let me tell you about a time when I moved to a new area as the senior product manager. The previous product manager left the company, so I picked up on a high visibility project that had already started - I had to get up to speed quickly and cut through ambiguity to figure out a solution.

Question 2

Whats the short version of the probem statement?

Answer 2

• Gap gross margin is -2% on the year, Gap internal Merchant’s are being challenged - are they really effective at determing what products should be sold where?
  
  • Merchants feel frustrated and don’t like like they have the data to ake these predictions.
• To equip merchants with the data to more accurately make predictions and deliver a better customer experience, a previous product manager led a project and a successful pilot to develop ML algorithm, and led a successfuly pilot, to better determine what products should be sold where. - which s exiciting!
• But here is the problem - this product manager left the company and I inherited the project at the critical time.
• The pilot for just 6 stores is successful and complete and Gap is excited. But where do we go from here?
• We used 3rd part data science firm to help us manually pick the right products to send to those 6 stores, how do we take the algorithm and make it usable for a regular merchant in Gap?

Question 3

What is the deep dive on the problem and the user?

Answer 3

User - the Gap Merchant.

• The merchant’s job is to determine what types of products should be sold in what stores. Gap knows what types of customers shop in each of their locations.. So the Gap Merchant is really trying to predict what types of products such as Men’s athletic wear, socks, pajamas, will be enjoyed by different types of customers, based on their respective store location.

Problem

• The merchant is using “last year performance” and intuition alone to preidct what types of products should be sold in what locations.
• As a result, Gap gross margin is down 2% year to date - a clear indicator that Gap merchants are inaccurate in determining what types of products their customers will enjoy.
• How are the merchants feeling? Merchants lack confidence in their predictions. They don’t feel like they have the tools to be successful

Question 4

What is the internal problem?

Answer 4

• For the solution to Gap’s problem, I inherited a project that was to work with a 3rd party data science firm to develop a machine learning algorithm that can determine what types of products should go to what locations based on a set of attributes, increasing gross margin and sales.
• The previous product manager and the 3rd party company had completed a small pilot of 6 stores that proved to increase gross margin and sales.

Confusion:

• Where do we go from here?
• There’s confusion and a lack of leadership internally on how to take a model that a 3rd party developed, and make it usable in Gap.
• How that we have a completed pilot, where do we go from here?

Question 5

So what did I do to cut through the ambiguity and deliver merchants the help they need?

Answer 5

Here are the actions that I took.

1. Rally the stakeholders - we need to gather our key stakehokders who are going to help us drive this project to completion, and rally them to this cause.
2. Align on Usability and Next Steps -

• How will merchants action?
  
  • Get to know the merchant
  • Merchant journey mapping
  • Why type of output can be actionable

1. Build the model in Gap - We need to build out the model in Gap to the aligned upon model output.
2. Test the model - Can we A/B test the model to see what output is effective?
3. Implement the model so it is actionable - let’s implement the model on a larger scale.
4. and Scale the model - Can we plan to implement the model to other brands outside of Gap?

Question 6

How did I rally the stakeholders?

Answer 6

Stakeholder alignment:

I’m new so..

1. I did stakeholder mapping to get to know who they key stakeholders were and to begin to make relationships.
2. Rallying stakeholders and getting them excited about the use cases for the model by:

• Socialize the pilot
• Setup a brainstorming session with key stakeholders to recap the pilot and discuss what we want to achieve.
• Present output of the discussion to senior leaders
• Scope out potential next steps.
• Steps:
  
  • Merchant works with design to determine how many items we need by hierarchy
  • Design presents to merchant to new product offerings for each hierarchy
  • Design Builds out a BOM
  • Merchants assign an item to a set of stores
  • Assortment is finalized and passed off to planning

1. We had an algorithm but needed to align on practical use cases for the model. How will merchants action?
   * Conclusion: We want to impact the second to last step of assigning an item to a set of locations.

Question 7

How did we algin on usability and next steps?

Answer 7

• Had a working group walk us through the current merchant process.
• With the model we can actually use it to improve multiple parts of the merchant process.
• We used a simple prioritization framework to evaluate the benefit, technical effort, and change management required.
• Process Steps:
  
  • Merchant works with design to determine how many items we need by hierarchy
  • Design presents to merchant to new product offerings for each hierarchy
  • Design Builds out a BOM
  • Merchants assign an item to a set of stores
  • Assortment is finalized and passed off to planning

1. We had an algorithm but needed to align on practical use cases for the model. How will merchants action?
   * Conclusion: We want to impact the second to last step of assigning an item to a set of locations.

Question 8

How did I approach build?

Answer 8

• We need to build the model in Gap
  
  • Build out the model in databricks azure
  • Rally cross-functional teams to be effective
    
    • Data science
    • Data engineering
    • Software engineering
    • Change Management
    • Product Management
    • Business Stakeholders
  • How can we build out a communication channel to ensure the model that we’re “tailoring” meets the use cases of the business
  • How can we showcase our efforts to the business to get feedback an iterate

Question 9

How did we test the model?

Answer 9

Test the model

1. Larger Pilot to run an A/B test - Can we test the output of the model in a set of stores?

• For A we had the merchants make predictions based on “last year performance” and their intuition.
• For B, we tested the model output in a series for one department, 46 store pairs. In that department we saw the following:
  
  • The results warranted a 1.6% GMROI lift
  • +2% Sales / traffic
• Issues:
  
  • Some stores didn’t get the inventory they needed due to supply chain issues so they couldn’t really satify the model.
  • The merchants used the model as a “recommendation” so didn’t necessarily follow it strictly for all stores.
  • We did find better results with the stores that did follow the recommendations more closely.

Question 10

How did I approach Implement?

Answer 10

• Implement an actionable output of the model that our merchants can use
  
  • How do we implement an output that is actionable?
  • With what groups do we start with?

1. Simple Output - Can we have merchant actually use a sample CSV file that gives them an output of optimizing what items should go to certain stores?

• This output was used in the pilot, but can we make it more user friendly so the entire brand can use it?
• Every week we ran the model in a simple reports and generate a CSV file.
• Shows items a merchant plans to “Buy”, and a recommendation of what store locations they should go to.

Question 11

How did we scale?

Answer 11

1. Scale within Gap
   
   1. Let’s implement to all areas of Gap (remaining departments and all store locations).
2. Make more user friendly
   
   1. Let’s make the output more user friendly for the merchant so they can action.
3. And scale the model to all “areas” of Gap.
   
   1. Let’s scale the model to Old Navy next, then consider Athleta.

Question 12

What results are we seeing?

Answer 12

• Results:
  
  • We have scaled toall department in Gap and all locations.
  • Since 3 months after go-live for all departments and all store locations we have the following results:
    
    • Results - quantified - +2% increase in Gross Margin Return on Investment, +1% in Sales Over Traffic.
• What could have gone better:
  
  • We needed to build a more robust store communication component that explains why the algorithm recommended a certain product to a store
    
    • Store managers are often lost and wondering why a certain product was recommended, and soetimes disagree.
    • Exisiting communication is just the CSV, and the merchant does their best to explain why.