How Google does Machine Learning Flashcards
What types of Vertex AI Workbench exist and what are the differences?
Managed instances – JupyterLab instances provisioned by Google, you can choose between different frameworks like Tensorflow or Pytorch, direct integration with many services like Dataproc, CloudStorage, BigQuery, define compute resources (CPU, GPU)
User-managed instances – JupyterLab instance that is highly customizable, you can use is as a Deep Learning VM, gives you more control of your environment and you can set it up within your VPC
If you change on step in the Vertex AI Pipelines or do have an error do you have to rerun the entire pipeline?
Vertex AI Pipelines are supporting caching of pipelines steps, it will rerun from the step where error occurred or for step that was modified.
What are best practices for storing the structured and unstructured data in ML?
Structured data should always be stored within BigQuery while unstructured data should be in Google Cloud Storage
What are recommended services to preprocess tabular and unstructured data?
Tabular data should be preprocessed within BQ if it can be done with SQL transformations. For unstructured data and transformations that are not available via SLQ you should use Dataflow. If client is familiar with Spark, you can use Dataproc for transformations.
Can Vertex AI work without managed datasets?
Yes, they are not mandatory.
What is the recommended next step after data preprocessing?
First do feature engineering and the create managed dataset in Vertex AI.
What should you use for data preprocessing if you are using TensorFlow framework?
TFX (TensorFlow Extended)
Depending on the size of the dataset when to train a model within JupyterLab notebook and when to use Vertex AI training?
For smaller datasets use only JupyerLab notebook.
What is one of the tools offered by GCP that supports you in developing ML systems in a responsible way?
What If Tool (free), which is part of Tensorboard (paid) - help you diagnose fairness issues in your data, in your labels, etc
How to make classification model more inclusive?
When you are satisfied with model performance, you can split the data is sub-groups (ex. by race, gender, etc) and validate performance per specific sub-group. You can use confusion matrix to calculate precision, recall, false positive rate and false negative rate. Choosing which rate to optimize and to which extent is highly dependent on the problem you are trying to solve and there is no easy answer.
What is equality of opportunity in responsible AI and equal opportunity threshold?
This is an approach that should be considered when designing an ML system that should make sure that you have an equal chance of getting the same prediction of the ML system regardless of the group you are part of. Threshold in this case is a cut-off (ex. financial income) where you classify all people above a certain threshold differently. It can often mean that threshold should have different values per sub-group to keep the same results.
What are Facets?
This is a an open-source product by Google that is helping you discover new insights from data, ex. detecting data skew, non-uniform feature distribution, detect differences in data between training and test set, etc.
