Earn Trust Flashcards
QUESTION: How do you earn the trust of stakeholders?
OR
What’s your strength?
Title: Getting it done right!
Situation:
I pride myself on getting it done, and getting it done RIGHT, whatever “it” may be.
Obstacle/Task:
Highly effective at listening, distilling and executing across a range of initiatives, products and projects
This translates to earning trust
Action:
“I’m involved in” high-level strategic initiatives requiring thinking big and outside the box
Exploratory effort w/ a large automotive manufacturer to strategize how it’s connected vehicle data providing GPS coordinates could pair with Veritone’s ad occurrence data to create a new partnered offering
Defining product strategy for the future of Veritone’s flagship application
Building an application from conception to launch and beyond
Low-level tactical efforts triaging bugs and troubleshooting problems
Looking into count discrepancies between customer provided records and those that appear within our application
Scaling myself out of a problem by creating reproducible steps and a playbook for others to leverage (troubleshooting advertiser connections)
Results:
Charting a course for blue-sky opportunities leading to forged partnerships
Penetrating new market segments, like audio and video based attribution, unlocking $4M in revenue and equipping customers with a solution
Decreased bug resolution time by 15% by partnering with devs to debug issues
Creating processes to enable others to replicate troubleshooting techniques
What this story demonstrates (skills, principles):
Earn Trust
QUESTION: Tell me about a time when you made a mistake.
Title: Meet at the customer, not the other way around (More than just API to send data)
Situation:
Upon product launch, only method to interface with this attribution service was via API
I had worked with the developer team to harden our GQL endpoints for security and reliability
However, most customers didn’t have a technical team familiar with APIs or tech savvy
Customer systems used antiquated software not compatible with modern solution
Caused a technical chasm for customers preventing value extraction from the product
Strain on Sales and Success as customer was frustrated
I had forced customer to come to us vs. other way around
Obstacle:
Determine to find alternative approach lessening the technical burden on the customer
Action:
Interviews to understand customer constraints (system incapability and technical competency challenges)
Led series of internal technical discussions to explore data delivery options
Challenged Eng proposal to install code, Raspberry PI, locally on the customer’s machine (escalated to CEO to reject proposal)
No mechanism to monitor code health locally
No resources to support a distributed code model
Advocated for a compatible solution to the customers’ systems and processes
Took customer requirements, oversaw development and built real-time (TCP Proxy) and batch services (CSV email delivery)
Results:
Expanded how customers could interface with my app (three options instead of one)
Unblocked 3 customers threatening cancellation due to onboarding difficulties
Reduced on-boarding time from 6+ weeks to than 4 weeks with other delivery options
When I introduced a CSV delivery it allowed for customer properties that I then mapped internally illustrating a lesson learned.
What this story demonstrates (skills, principles):
Are Right, A Lot
Earn Trust
QUESTION (2nd): Tell me about a time when you made a mistake.
Title: Don’t count that city twice!
Situation:
Product that provided a geographical lens by Nielsen’s definition of a Designated Market Area (DMA)
Programmatically mapped cities to counties to DMA (city > county > DMA) across two disparate data sources (Google Maps for city and Nielsen for County/DMA)
In cases where Nielsen splits a county (east, west, north, south), I was unable to programmatically determine which region to assign the city so I include it in both regions
Obstacle/Task:
Customer report a location data issue where by a city was accounted for in multiple DMAs
For example, Rio Vista appeared in both ‘Sacramnto-Stkton-Modesto’ and ‘San Francisco-Oak-San Jose’ DMAs because of the underlying data structure
Action:
Investigated the data relationship. Rio Vista is a city within Solano County. However, Nielsen splits Solano County into east and west.
Illogical for a city to be shared on both sides of the county but is an artifact of how Nielsen split (east, west, north, south) and programmatically mapped a city to both regions of the county.
Script to flag all cases where a city was shared in two DMAs due to the split
Manually audited and assigned city to one and only one side of the county
Results:
Resolved location discrepancy issue by reloading the manually audited data
Recognition that an initial audit following the programmatic assignment of cities would have revealed this obvious underlying issue
Principle action to review work once complete
What this story demonstrates (skills, principles):
Bias for action
Earn Trust
QUESTION: Tell me about a time when you worked on a project that got delayed. How did you handle it? Once you know it would be delayed, explain the actions you took afterwards.
Title: How do you eat an elephant (piece by piece)?
Situation:
Galvanize a team by conveying a strong vision and picture of success
Organically drive passion and velocity through effective communication
Positioning individuals and teams for success; delivering for the customer
Tasked with fulfilling a contractual client roadmap containing 36 small to large features and capabilities
Business expectation to complete work within 1 quarter (w/o product input)
Unable to bill customer until list is complete and deemed “product ready”
Customer planning to terminate competitor’s service and transition asap
Obstacle:
Other teams, like customer success, mobilizing around completion date
Team was discouraged by an unrealistic timeline
No change in resourcing or headcount
Business pressure to generate revenue
Action:
Deconstructed features and developed a matrix with each functional team
Design, Front-end, API, Engine, Database, QA
Hosted several full day working sessions with QA, Dev and Designers
Each respective workstream provided estimates
Arrived at a 9 month estimate
Achieved buy-in from functional owners tied back to accountability
Communicated revised timeline to respective stakeholders to reset expectations
Results:
Finished one month behind schedule due to customer scope creep; however, expectations were adjusted to this outcome
Incrementally introduced features when ready building team confidence
Capabilities enriched the product and benefited the broader customer base
Co-founder acknowledge team’s contribution during all hands
Activated second largest contract in company history
Empowered team through vision, accountability, communication, and execution
What this story demonstrates (skills, principles):
Hire and develop the best.
Earn Trust. Allowed everyone on the team to contribute to the timeline
Frugality
QUESTION: Tell me about a challenging client-facing situation and how you handled it.
Title: Hothead Fede
Situation:
A year ago, an industry leading audio broadcaster enlisted its Research team to oversee the transition from a competitive offering to my application, Attribute
Involved on-site, weekly working meetings and a comprehensive product tear-down to identify roadmap requirements necessary to transition
A lot of pressure on both sides to ensure the transition happens as quickly as possible pending “product readiness”
Economics were more favorable on my product and solution was richer
Used a shared Trello board as a collaborative workspace to capture requirements and track feature progress
Obstacle/Task:
During a working meeting of requirements gathering, I capture the requirements described by the customer and summarized in an email
Several weeks later when providing a development update on the requirement, the customer ignites and uses expletives saying that we’re building it wrong
Action:
I referenced the email summary and requirements, but that did little to diffuse his anger
I seeked clarity on the requirement revisions
I suggested we reconvene after I huddle internally to determine LOE
I partnered with my GM for support on a go-forward basis to participate in client meetings to intervene if the customer became unruly
Informed customer that we could accommodate the ask but need to adjust the timeline by two weeks
Introduced a more rigorous requirements gather process where all requirements were captured in a shared doc and required the customer’s sign-off before work could start
Advocated that a PM on the customer side help shape requirements and work directly with me to facilitate the customer needs
Results:
Tighter alignment across parties, better understood requirements and a healthier working relationship
Customer’s PM helped steer the unruly individual and reference the agreed upon requirements if there was oscillation
Recognition that there was a lot of responsibility on the Research team and I had to earn their trust that I could get it right even if the requirements changed
What this story demonstrates (skills, principles):
Earn trust. Listen and be self critical. Remained professional
QUESTION: Tell me about a time when you had a problem and you had to go through several hoops to discover the root cause.
Title: Follow the data trail
Situation:
Attribution solution relies on customer ad metadata and correlates it to advertiser website to measure advertising response
End users, typically media sellers, intimately know their campaign makeup and how many ads aired/delivered
In early-2019, shortly after product launch customers reported ad count discrepancies
Example: a campaign is comprised of 100 ads, but only 77 appeared in the application
Obstacle/Task:
Count differences eroded customers’ trust in the product and data integrity concerns threatened the users confidence
Experienced customer terminations and at-risk accounts
Action:
Requested customer counts to compare to app counts and saw a 23% avg. discrepancy
Diagramed the data flow to identify potential points of failure in the pipeline
Customer > Database > Discovery > Attribute
Sequentially analyzed total counts in each category
Customer = database; assurance that provided data was written and stored properly
Count difference between database and Discovery; isolated service failure
Discovery and Attribute at parity; confidence in interoperability between apps
Guided support teams fielding customer complaints to structure information with this flow to identify patterns with the same services
Established a “recovery plan” and worked with technical teams to implement retry logic on failing API and correlation services
Advocated for platform team to repair underlying services
Implemented retry logic and post-process to sweep for failures and correct problems
Results:
Speed and confidence around the issue; educating support teams on how to structure information enabled me to better validate a systematic platform issue
Pinpointed root cause and guided the Platform team to the issue
Retry and post-process services resolved 95% of discrepancy reports in 2 months
What this story demonstrates (skills, principles):
Deep dive
QUESTION: Describe a time when you had to communicate a complicated topic to a group of people completely unfamiliar with the topic?
Title: No (geo) cross pollination allowed
Situation:
A feature of my attribution solution enabled users to set geofences to effective target and include web traffic from areas from outside the broadcast
Ideal for adjacent markets such as a broadcast that originates in LA but may spill into SD
However, a large national customer expressed concern with how geofences operate when looking at multi-market campaigns vs. individual market campaigns for the same markets
Visits that occur in one market (e.g., San Diego) were applied to an ad in another market that may or may not be geographically close (e.g., Chicago) just because it qualified within the attribute timespan (e.g., 10 mins). The byproduct of this logic is an increase in Ad Visits that elevate the campaign performance due to visits occurring from markets outside of the station’s broadcast.
The geofences effectively qualified what visits were candidates to become ad visits regardless of their location. The existing geofence design allows users to specify locations of inclusion; however, this flexibility had undesirable side effects.
Obstacle/Task:
I wanted to validate the existing logic was working as designed
I wanted to clearly communicate the behavior so I could work with the customer to implement something that aligned with its expectations
Action:
I created a multi-market campaign comprised had 8 individual markets from across the country and then created 8 individual single market campaigns with identical criteria to compare the counts in a spreadsheet
I worked with the developer to validate that the increased count was not the result of a bug but rather the designed logic
I illustrated three scenarios via a diagram for the customer given the complexity with the goal of referencing the visuals to assist in simplifying the explanation
Current, proposed and not supported states
Overlaying commentary to the visuals was effective in describing the issue
Request to revise logic to ensure that visits that occur in one market will be disqualified as Ad Visits in another market for which there are broadcast ads
This new logic extends to adjacent markets thereby disqualifying any visits that originates within them from being applied to their contiguous market.
To illustrate, a geofence in Dallas will only acknowledge Ad Visits that originate within Dallas even if an adjacent geofence is set for the Austin market. There will be no cross-pollination.
On occasions whereby a local campaign requires the inclusion of markets outside of the broadcast, iHM users will remove the geofence. This can be performed by clearing all geofences applied to the campaign OR selecting ‘No’ for “Is this advertiser in multiple markets”?
Captured requirements in a change order document requiring customer signature to advance
Modified the logic to align with customer expectations
Results:
Achieved parity between single and multi-market campaigns with exact same ad visit counts
Eliminated discrepancies that artificially inflated the campaign performance metrics resulting from multiplying ad visits by each available market
Collaborated with the customer to achieve a solution that met its needs and improved the product
Deployed fix within two weeks of discovery
What this story demonstrates (skills, principles):
Earn Trust
Deep dive
Invent and simplify
QUESTION: When was the last time you had to apologize to someone?
Title: I’m sorry Attribute Roundtable
Situation:
Application has deep dependencies on platform
Responsible for data ingestion, storage, eventing and correlation
Numerous components involve and failures across the pipeline were not uncommon
Failures results in count discrepancies diminishing product value (customer sent 10 records, my product showed 7)
Experienced customer complaints and confusion regarding discrepancies
Vulnerable Sales and Success teams forced to handle upset customer and product problems
Morale was low and teams were frustrated and fatigued from on-going problems
Obstacle/Task:
Platform team prioritized building a new framework and didn’t want to invest in fixing the existing pipeline issues
2 non-converting trials and 2 customers cancellations citing data inaccuracies and quality concerns resulting in lost revenue
Action:
Deep apology to cross-functional team during time of crisis (sales, product marketing, customer success, stakeholders)
Pitched a “recovery plan” to executives necessary to rightsize the issue
Diagram process flow and components prone to failure (adapter > api > engine/eventing > index > engine > app)
Components were absent of retry logic; implemented exponential backoff up to 3 max for system resiliency
Stood up dashboards for monitoring data flow and alerting of failures
Deployed a post-process reconciliation service that addresses hourly failures
Results:
Reduced errors from 23% to 12% retry logic
Reduced discrepancy related tickets by 95% within 2 months with post process service
Improved customer satisfaction and avoided discrepancy related cancellations
Alleviated stress points on sales and support teams
Restored team and customer confidence in product
Enabled platform team to focus on new framework in order for better system reliability
What this story demonstrates (skills, principles):
Earn Trust
Have backbone; disagree and commit
