Bias for Action Flashcards
QUESTION: Tell me about a time when you had a conflict and you had multiple ways to resolve it. How did you evaluate your options? Which solution did you choose and why?
Title: Automation or Traffic System
Situation:
2.5 years ago; developing near-real time attribution solution that required broadcaster metadata
Important to the product because it provide the data and time of ad event
Metadata capture could be done in two ways
Automation system
Traffic system
Obstacle/Task:
My goal was to disambiguate between the two systems and select an integration method
Action:
Conducted interviews with broadcasters to understand the nuance and benefits
Evaluated the trade-offs based on delivery mechanics, speed, and data composition/makeup, and customer value
Traffic:
Housed only ad event metadata (only advertisements, not program content), which was the requirement for the product
Fed ad event data into the automation system for playout management
Involved manual or automated reconciliation process that took place 24-48 hours after an ad event aired to ensure data integrity
Post-process, batch data delivery through usually through a packaged CSV emailed to the system
An entire day of ad events were a delivery payload
Automation:
Orchestrated the ad, music and content playout (holistic asset management, not just ads)
Events emitted in real-time
Real-time nature doesn’t allow for reconciliation, which can cause up 5% discrepancy between what the system thinks aired vs. what actually aired
Leverages real-time data transmission through API
Each event was a discrete payload
Pursued automation system for holistic data capture, real-time and API delivery despite minimal data discrepancy
Near real-time insights was a competitive advantage enabling the customer to act on data faster not found in other offerings
Expanded data capture beyond just the ad which had long-term strategic value (correlate metadata to actual broadcast for data validation and derivative products)
Results:
Unlocked near-real time value, but faced data discrepancies that were difficult to triage
Customers reported count differences up to 10% likely due to failed dispatched events from the broadcaster or unreceived on my end due to API failures
Eventually integrated traffic systems and benefited more accurate data due to reconciliation process and reduced data transmission complaints to less than 5%
Customer use case is largely post-analysis vs. real-time
What this story demonstrates (skills, principles):
Bias for action
QUESTION: Tell me about a time when you made a decision too quickly and what you would have done differently.
Title: Don’t count that city twice!
Situation:
Product that provided a geographical lens by Nielsen’s definition of a Designated Market Area (DMA)
Programmatically mapped cities to counties to DMA (city > county > DMA) across two disparate data sources (Google Maps for city and Nielsen for County/DMA)
In cases where Nielsen splits a county (east, west, north, south), I was unable to programmatically determine which region to assign the city so I include it in both regions
Obstacle/Task:
Customer report a location data issue where by a city was accounted for in multiple DMAs
For example, Rio Vista appeared in both ‘Sacramnto-Stkton-Modesto’ and ‘San Francisco-Oak-San Jose’ DMAs because of the underlying data structure
Action:
Investigated the data relationship. Rio Vista is a city within Solano County. However, Nielsen splits Solano County into east and west.
Illogical for a city to be shared on both sides of the county but is an artifact of how Nielsen split (east, west, north, south) and programmatically mapped a city to both regions of the county.
Script to flag all cases where a city was shared in two DMAs due to the split
Manually audited and assigned city to one and only one side of the county
Results:
Resolved location discrepancy issue by reloading the manually audited data
Recognition that an initial audit following the programmatic assignment of cities would have revealed this obvious underlying issue
Principle action to review work once complete
What this story demonstrates (skills, principles):
Bias for action
Earn Trust
QUESTION (2nd): Tell me about a time when you made a decision too quickly and what you would have done differently.
Title: Orphaned Advertisers
Situation:
Last year, I introduced a first of its kind user permissions model at Veritone that assigned an object (adv/campaign) to its creator
Additionally, supported sharing functionality for peer visibility along with the ability to transfer ownership
Model ensured that users created objects remained private to them unless shared
Once in production, experienced questions from Success regarding the advertisers of deactivated users
Obstacle/Task:
I overlooked the scenario for when an owner of the object (adv) is deactivated (no longer with the organization) and how to handle this in the UI
Action:
Replicated the scenario to assess the existing behavior before engineering a solution
Reviewed database and identified 10 advertisers in 3 orgs with an deactivated user
Designed a workflow that would categorize this type as ‘Unassigned’ advertiser
Made ‘Unassigned’ folder visible to any ‘Manager’ user type for assignment to a) themselves or b) another user in the system
Validated the concept with internal and external users
Results:
a) successful in creating UI visibility to unassigned advertisers b) enabled managers to assign a user
Reduced the number of unassigned advertisers by 50% within one weeks
Experience forces me to think about “referential integrity” between primary and foreign objects and how changes to primary object without addressing the foreign object can result in orphaned objects when writing requirements
Ingrained in train of thought in product development and applied to test cases to better account for scenarios
What this story demonstrates (skills, principles):
Learn and be curious
Bias for action
QUESTION: Adjust work priorities to meet changing demands?
OR
Tell me about a time when you had to leave a task unfinished.
Title: Introduction of Google Analytics 4
Situation:
In Q4-20, focused on completing my end of year roadmap
Enhancing the pre-campaign search previewer so users would have more helpful insights when building a campaign
However, in late October last year, Google Analytics announced the introduction of GA4 causing me to shift focus
At the time, Google does not have commercially available APIs for any third party to interface with its GA4 service
Existential blocker preventing GA4 accounts from interfacing with my product
Advertisers that upgrade or create a new account were unable to connect
Obstacle/Task:
Goal to provide internal and external parties with guidance during time of influx
Action:
I discovered Google had an Analytics Data API Trusted Testers program to gain early access to its alpha API
Reviewing it’s API Support docs, the endpoint doesn’t provide granular data needs (hour and minute level records)
Another existential threat
Collaborated with Google to identify alternative pathways to run GA-U and GA4 concurrently
Crafted help article outlining connection options and vetted with internal stakeholders before socializing
Proactively prompted advertisers hitting invitation landing page with guidance
Results:
Mitigated the perception that it was a “product problem”
Provided an alternative method for advertisers to connect
Mobilzed quickly around an urgent matter and identified alternative support methods within 2 weeks
Connected 3 advertisers that otherwise would have been at an impasse
No option but to shift priorities given the severity of the change
What this story demonstrates (skills, principles):
Ownership
Bias for action. Speed matters, quick responsiveness to something that may have been existential threat
