All

Question 1

For all non-numeric columns other than timestamp, BigQuery ML performs a one-hot encoding transformation. This transformation generates a separate feature for each unique value in the column.

Answer 1

True

Question 2

To understand other performance metrics, you can configure Google’s Cloud monitoring to monitor your model’s traffic patterns, error rates, latency, and resource utilization. This can help spot problems with your models and find the right machine type to optimize latency and cost.

Answer 2

True

Question 3

What is the trade-off between static and dynamic training?

Answer 3

• Static is simpler to build and test, but will probably become stale.
• Whereas dynamic is harder to build and test, but will adapt to changes. Part of the reason the dynamic is harder to build and test is that new data may have all sorts of bugs in it.

Question 4

What are three potential architectures to explore for dynamic training?

Answer 4

• Cloud Functions,
• App Engine,
• Dataflow.

Question 5

Describe In a general architecture for dynamic training using Cloud Functions.

Answer 5

• a new data file appears in Cloud Storage,
• the Cloud Function is launched.
• the Cloud Function starts the AI Platform training job.
• the AI Platform writes out a new model.

Question 6

Describe In a general architecture for dynamic training using App Engine.

Answer 6

• when a user makes a web request from a dashboard to App Engine, an AI Platform training job is launched,
• AI Platform job writes a new model to Cloud Storage
• from there, the statistics of the training job are displayed to the user when the job is complete.

Question 7

Describe how the Dataflow pipeline can be invoked the model for predictions.

Answer 7

• the streaming topic is ingested into Pub/Sub from subscribers.
• Messages are then aggregated with Dataflow,
• aggregated data is stored in BigQuery.
• AI Platform is launched on the arrival of new data in BigQuery,
• then an updated model is deployed.

Question 8

How the latency can be improved when serving models?

Answer 8

• Static serving then computes the label ahead of time and serves by looking it up in the table.
• Dynamic serving in contrast computes the label on demand.

Question 9

Describe a space-time trade-off in serving prediction model.

Answer 9

• Static serving is space intensive, resulting in higher storage costs because we store precomputed predictions with a low, fixed latency and lower maintenance costs.
• Dynamic serving, however, is compute intensive. It has lower storage costs, higher maintenance, and variable latency.

Question 10

What is Peakedness in a data distribution?

Answer 10

Peakedness in a data distribution is the degree to which data values are concentrated around the mean, or in the case of choosing between model serving approaches, how concentrated the distribution of the prediction workload is.

Question 11

What is Cardinality in a data distribution?

Answer 11

Cardinality refers to the number of values in a set. In this case, the set is composed of all the possible things we might have to make predictions for.

Question 12

When to choose static vs. dynamic model serving?

Answer 12

• When the cardinality is sufficiently low, we can store the entire expected prediction workload.
• When the cardinality is high because the size of the input space is large and the workload is not very peaked, you probably want to use dynamic training.
• In practice though, a hybrid of static and dynamic is often chosen, where you statically cache some of the predictions while responding on demand for the long tail. This works best when the distribution is sufficiently peaked.

Question 13

What design changes need to be made If you want to build a static serving system?

Answer 13

• you need to change your call to AI Platform from an online prediction job to a batch prediction job.
• you need to make sure that your model accepted and passed through keys as input. These keys will allow you to join your request to prediction at serving time.
• you write the predictions to a data warehouse like BigQuery, and create an API to read from it.

Question 14

Explain Extrapolation and Interpolation.

Answer 14

• Extrapolation means to generalize outside the bounds of what we’ve previously seen.
• Interpolation means to generalize within the bounds of what we’ve previously seen.

Question 15

How can you protect a model from changing distributions?

Answer 15

• be vigilant through monitoring (summary statistics of inputs over time)
• check residuals (the difference between its predictions and the labels, have changed as a function of your inputs)
• emphasize data recency during model training
• retrain the model frequently

Question 16

Describe types of drift in ML models.

Answer 16

• Data Drift or change in probability of X P(X) is a shift in the model’s input data distribution.
• Concept drift, or change in P(Y|X), is a shift in the actual relationship between the model inputs and the output
• Prediction drift, or change in the predicted value of Y given X, is a shift in the model’s predictions.
• Label drift or change in the predicted value of Y as your target variable is a shift in the model’s output or label distribution

Question 17

List changes in the data distribution of the inputs.

Answer 17

• Data drift,
• feature drift,
• population,
• covariate shift

Question 18

Define concept drift in ML model.

Answer 18

Concept drift occurs when the distribution of our observations shifts over time, or that the joint probability distribution we mentioned before changes.

Concept drift can occur due to shifts in the feature space and/or the decision boundary, so we need to be aware of these during production.

Question 19

What if you diagnose concept drift?

Answer 19

If you diagnose concept drift, the old data needs to be relabeled and the model retrained.

Question 20

What if you diagnose data drift?

Answer 20

If you diagnose data drift, enough of the data needs to be labeled to introduce new classes and the model retrained.

Question 21

Why and when does distribution skew occur?

Answer 21

Distribution skew occurs when the distribution of feature values for training data is significantly different from serving data and one of the key causes for distribution skew is how data is handled or changed in training vs production.

Question 22

What is TensorFlow Data Validation?

Answer 22

TensorFlow Data Validation is a library for analyzing and validating machine learning data, for which there are three components:
- The Statistics Generation component ( generates features statistics and random samples over training data, which can be used for visualization and validation)
- the Schema Generation component,
- the Example Validator component.

Question 23

What the SchemaGen pipeline component is for?

Answer 23

A SchemaGen pipeline component will automatically generate a schema by inferring types, categories, and ranges from the training data.

Question 24

What ExampleValidator pipeline component is for?

Answer 24

The ExampleValidator pipeline component identifies anomalies in training and serving data.

The ExampleValidator pipeline component identifies any anomalies in the example data by comparing data statistics computed by the StatisticsGen pipeline component against a schema.

Question 25

TensorFlow Data Validation is a component of TensorFlow Extended, and it helps you to analyze and validate your data. Data validation checks include identifying feature correlations, checking for missing values, and identifying class imbalances.

Answer 25

True

Question 26

What does training-serving skew refers to?

Answer 26

Training-serving skew refers to differences caused by one of three things:
- A discrepancy between how you handle data in the training and serving pipelines.
- A change in the data between when you train and when you serve,
- A feedback loop between your model and your algorithm. Up until now,

Question 27

What is QPS?

Answer 27

The performance consideration is not how many training stamps you can carry out per minute, but how many queries you can handle per second. The unit of this, queries per second, is often called QPS.

Question 28

What do you need to build hybrid machine learning systems?

Answer 28

In order to build hybrid machine learning systems that work well both on-premises and in the cloud, your machine learning framework has to support three things:
- composability,
- portability,
- scalability.

Question 29

What technologies are important in hybrid machine learning systems?

Answer 29

• Kubeflow (offers portability and composability between your on-premises environment and Cloud ML Engine)
• TensorFlow Lite (TensorFlow Lite makes specific compromises to enable machine learning inference on low-power or under-resourced devices)

Question 30

Is Kuberflow serverless?

Answer 30

No
You will have to do cluster management.