Launching into Machine Learning

Question 1

Can you use categorical features from ML model training?

Answer 1

You can use them directly, you have to convert it to a numerical representation (ex. one-hot or multi-hot encoding)

Question 2

What the 2 common types of EDA?

Answer 2

Univariate – explore only one feature and value distribution
Multivariate - compare multiple features and relation

Question 3

What type of graphs are used during EDA?

Answer 3

Histograms, Scattered graphs and Heat maps

Question 4

List some examples of Data Quality

Answer 4

Question 5

What are 3 basic steps in EDA?

Answer 5

1. Understand the data
2. Clean the data
3. Analysis of relationship between variables

Question 6

What is MAE, MSE, RMSE and their differences?

Answer 6

MAE – average error
MSE – average squared error, better than MAE as it punishes larger errors
RMSE – better than MSE as it displays error in the predicted unit (ex. deviation of basketball point is 4 instead of 16 in MSE)

Question 7

Why do we have regularization in logistic regression?

Answer 7

• Gradient vanishing or exploding problem
• Keep logits stay away from asymptotes which can halt the training

Question 8

How to prevent overfitting?

Answer 8

Regularization and early stopping

Question 9

Is confusion matrix available in AutoML?

Answer 9

They are available for classification model, but you need to have 10 or fewer values for the target column

Question 10

Which type of data is AutoML supporting?

Answer 10

Tabular, Image, Video, Text

Question 11

What is the maximum amount of time steps that AutoML Forecast is supporting?

Answer 11

3000

Question 12

What is data leakage?

Answer 12

Data leakage is when you are using a feature that is highly correlated to the target you are trying to predict but that feature is not available during inference (ex. predict if a customer will sign-up and you are using his sign-up payment transaction for training). Model will have high performance during testing but will most probably not perform that good when it is deployed.

Question 13

What is training-serving skew?

Answer 13

Training-serving skew is when input features used during training are different from features available during model serving (ex. train model with hourly data but only weekly data is available during model serving)

Question 14

If precision and recall are good for a certain threshold for all classes (in case of multi-classification problem) except one, what would be your approach to resolve this?

Answer 14

You can change the threshold only for that one class.

Question 15

For AutoML image and video, what is the minimum amount of videos/images of a certain class compared to amount of videos/images of other classes?

Answer 15

Class with lowest amount of videos should have at least 10% of training examples compared to a class with highest amount of videos.

Question 16

What is necessary to do when it comes to video data preparation?

Answer 16

You need to assign bounding boxes (if needed) and classes (ex. select a ball and assign a label “ball”)

Question 17

What parameters are used in AutoML video?

Answer 17

Frame rate – important for motion changes (ex. slow walking with low FPS can look like running)
Resolution – important for object tracking, recommended resolution is 256p
Prediction type – what are you trying to predict

Question 18

What type of problems are you able to resolve with Tabular AutoML?

Answer 18

Binary classification, multi-class classification (predict one out of more than 3 classes), regression, forecasting

Question 19

What solutions are available to resolve forecasting problems?

Answer 19

AutoML forecasting, BigQuery ML forecasting with ARIMA +, Forecasting with Prophet

Question 20

What models are available within AutoML forecasting?

Answer 20

• Temporal Fusion Transformers (TFT) – attention-base DNN model
• Time Series Dense Encoder (TiDE) – based on encoder-decorder model. Fast training and inference especially for long contexts and horizonts
• AutoML (L2L) – good for wide variety of use cases
• Seq2Seq+ - good for initial experimentation as it has a simple architecture. Works good with shorter time budgets and data size up to 1GB.

Question 21

When to use Big Query Arima Plus?

Answer 21

When you need fast training with many quick iterations and creation of inexpensive baseline model. It uses ARIMA in the background – univariate forecasting model.

Question 22

What is the difference between Prophet and BQ ARIMA PLUS?

Answer 22

Both of them attempt to decompose time-series into trends, seasonality, holidays and combine these models predictions, Prophet is trying to fit a curve using linear or logistic model. Benefit compared do ARIMA PLUS is that you can select hardware that is used for training.

Question 23

Is Prophet univariate or multi-variate model?

Answer 23

It is a multi-variate model but GCP is offering only univariate version of it.

Question 24

What types of predictions are available for forecasting models?

Answer 24

Only batch prediction.

Question 25

What are Tabular Workflows?

Answer 25

They are pre-build MLOps pipelines within Vertex AI Pipelines (Kubeflow templates) that can be used to solve problems like:
- Feature Engineering (detect most important features and rank them and transform features to ensure consistent input for training and inference)
- Auto ML – pipelines for using AutoML models
- TabNet - pipeline for TabNet architecture (uses sequential attention)
- Wide & Deep - pipeline for using Wide & Deep architecture (jointly trains wide linear models and deep neural networks)

Question 26

When to use CSV and when Big Query as a data source for your model training?

Answer 26

Depending where you data resides in, but in general, if it doesn’t require nested fields and more complex structures CSV file is enough.

Question 27

How can you inform Vertex AI that one feature is more important than another?

Answer 27

By default, all features are equally important during training. You can manually add weight as a number between 0-10.000 to rank the importance of features. If you add them manually, you must do it for all features.

Question 28

Which AutoML tabular problem model you can export and deploy on on-premise infrastructure?

Answer 28

Both regression and classification but not forecast

Question 29

Is Vertex Explainable AI support for exported table models?

Answer 29

No, you have to serve them within Vertex AI

Question 30

What is a context window in time series?

Answer 30

How far back in the past from the current data point in time series will a model look into to create a prediction.

Question 31

What is forecast horizont in time series?

Answer 31

How far in the future will a model predict.

Question 32

How can you automatically generate Transformations in AutoML forecast?

Answer 32

Run Generate statistics.

Question 33

Difference between attribute and covariate value in forecasting?

Answer 33

Attribute represents a static value that doesn’t change over time. It can be treated like metadata that just gives additional information around the data point itself. (ex. in stock price prediction for a company, attributes are company location, sector, name, owner). Covariate at forecast time is a leading indicator. Prediction data has to be provided for each point in the forecast horizont. This attributes are highly correlated with the target value (ex. suppose you want to forecast daily electricity consumption for a city. The electricity price, temperature, and day of the week can be considered covariate features)

Question 34

What is hierarchical forecasting and its purpose?

Answer 34

Lets say you want to forecast sales of the products. Products are organized into category hierarchy. Forecast on a category level and on a product level should add up. The purpose of this is to reduce bias. There are three types of biases: total, temporal and group level bias.

Question 35

What types of data splits are available?

Answer 35

Random split: define percentages that add up to 100% (default is 80/10/10 for traing/validation/test)

Manual split: add column where every row contains following case-sensitive values TRAIN, VALIDATE, TEST

Chronological split: Split based on time based column where earliest rows will be used for testing, next rows for validation and latest rows for testing

Question 36

What is model evaluation slice?

Answer 36

Model evaluations of a specific class or a label.

Question 37

How can you use your AutoML models on Android and iOS devices?

Answer 37

You need to export your model to a TensorFlow Lite (Android) or Core ML (iOS) format and integrate it within your application.

Question 38

What are 4 basic commands in BQ ML?

Answer 38

• CREATE OR REPLACE MODEL()
• OPTIONS()
• EVALUATE()
• PREDICT()

Question 39

What models and features a supported in BQ ML?

Answer 39

Question 40

What options are available to use Explainable AI in BQ ML?

Answer 40

EXPLAIN_PREDICT() – for each prediction what feature contributed the most
GLOBAL_EXPLAIN() – create feature attributions (you have to set enable_global_explain option to TRUE when training the model)

Question 41

When creating a matrix factorization model to create a recommender system in BQ ML, what kind of feedback you can use?

Answer 41

Explicit (rating) or implicit (how long did the user stay on pdp)

Question 42

What option you must use to use hyperparameter tuning in BQ ML?

Answer 42

num_trials – how many models will be created (each model has different combination of hyperparameters)

Question 43

Explain how hyperparameter tuning works.

Answer 43

For hyperparameter that you want to configure, you are putting from-to values where for most hyperparameter value combination it will train a new model. Number of trained models depends on num_trials parameter.

Question 44

What is a prerequisite for creating matrix factorization model in BQ ML?

Answer 44

You need to define slot reservations as this algorithm might be computationally intensive.

Question 45

What is cross-validation approach that can be used when there is not much training data?

Answer 45

Every data point is important so use a cross-validation – split on training and validation dataset multiple times.

Question 46

How do you do a train, validation, test split in BQ?

Answer 46

You have to generate a unique id of each row, for example create a unique hash our of data or some id column or a combination of columns by using FARM_FINGERPRINT() method. FARM_FINGERPRINT() method generates a unique INT64 number that is always the same for the same input value. When taking moduo of a specific number like 10, you are splitting dataset into 10 buckets which you can further narrow down.