Prioritizing Projects

Question 1

What are the cost drivers?

Answer 1

A pyramid:
The biggest one is the data availability - including how much data is needed, is the data stable, security, how expansive to label and get the data

The second one is accuracy requirements - what is the cost of wrong predictions? How frequently does the system needs to be right to be useful?
Ethical implications?

The last one is the how difficult is the problem - is it well defined? Published work on similar problems? Computer requirements? And if non of the above than it list can a human do it?

Question 2

What different archetypes does a machine learning project has?

Answer 2

1. Improve in existing code /part of a system that is govern by rules and we improve the rules with machine learning - exp: improve code completion in ide, video game AI
2. Help the human in the loop - make a prediction that the human will go through and see if he takes it. Help radiologist, email auto completion etc…
3. Autonomous systems - the system is making its own decisions in the real world. Like autonomous cars, automated costumer support.

Question 3

What is a data flywheel?

Answer 3

More users ➡️ more data ➡️ better model 🔄

If the user creat more data, and this data can be used to make better predictions, and the better predictions attract more users than we have this loop and it’s great :)

Question 4

Product designs considerations for human in the loop

Answer 4

Looking in a feasible on impact chart,
Product design can make them
Both better.
It can reduce the need for accuracy of designed well, so more feasible

Apple guidelines:
1. What role does the ml play in your app? - critical or complementary, private or public? Proactive or retroactive? Visible/invisible, dynamic/static
(Maybe make it less of a role to make it more feasible)
2. How can you learn from your users?
- explicit feed back, implicit feedback (“like this song”), calibration during setup (scan face for faceid), corrections (fix mistakes the model makes)
3. How should your app handle mistakes?
- limitations (show your users where you expect the model to work well), corrections (let users succeed even if the model fails), attributions (help user understand where suggestions come from), confidence (share the quality of results)