Deploy and retrain a model (10–15%)

Question 1

What things does MLflow track?

Answer 1

Everything, divided into parameters, metrics, and artifacts

Question 2

If I want mlflow on azure what package should I install?

Answer 2

pip install mlflow azureml-mlflow

Question 3

How to use mlflow on a local device?

Answer 3

Use the MLflow tracking URI from the overview of the workspace, then do mlflow.set_tracking_uri = “MLFLOW-TRACKING-URI”

Question 4

How to start a run in mlflow?

Answer 4

It acts like a wrapper, so import it then do
mlflow.set_experiment(experiment_name=<experiment>)</experiment>

Question 5

How to use autolog in MLflow?

Answer 5

from xgboost import XGBClassifier

with mlflow.start_run():
mlflow.xgboost.autolog()

model = XGBClassifier(use_label_encoder=False, eval_metric="logloss")
model.fit(X_train, y_train, eval_set=[(X_test, y_test)], verbose=False)

Question 6

What are some common function for custom logging in mlflow?

Answer 6

log_param(): single key-value parameter, log_metric(): single key-value metric, log_artifact(): logs a file, like an image, log_model(): logs a model as an MLflow model

Question 7

How to use custom logging in MLflow?

Answer 7

from xgboost import XGBClassifier
from sklearn.metrics import accuracy_score

with mlflow.start_run():
model = XGBClassifier(use_label_encoder=False, eval_metric=”logloss”)
model.fit(X_train, y_train, eval_set=[(X_test, y_test)], verbose=False)
y_pred = model.predict(X_test)

accuracy = accuracy_score(y_test, y_pred)
mlflow.log_metric("accuracy", accuracy)

Question 8

What is an endpoint?

Answer 8

An HTTPS endpoint to which you can send data and which will return a response

Question 9

What are the two types of online endpoints within Azure Machine learning?

Answer 9

There are managed online endpoints and Kubernetes online endpoints

Question 10

What are managed online endpoints?

Answer 10

Azure Machine Learning manages all the underlying infrastructure

Question 11

What are kubernetes online endpoints?

Answer 11

Users manage the kubernetes cluster which provides the necessary infrastructure

Question 12

What four things do you need to deploy a model to a managed online endpoint?

Answer 12

Model assets, scoring script, environment, compute configuration

Question 13

What is automatically generated when you deploy MLFlow models to an online endpoint?

Answer 13

The scoring script and environment are automatically generated

Question 14

What is blue/green deployment?

Answer 14

It’s just a/b testing. Set 90% of the traffic for the proven endpoint and 10% for the new one and see how it performs.

Question 15

What do you use to create an online endpoint?

Answer 15

You use ManagedOnlineEndpoint with the name and auth_mode parameters

Question 16

What do you need to deploy an MLflow model?

Answer 16

You either need local model files or a registered model in AML. You also need the instance_type and the instance_count

Question 17

How can you test an endpoint in the AML?

Answer 17

Go to endpoints and then test. Use your data and get the test result.

Question 18

How can you test an endpoint in Python SDK?

Answer 18

Send your data through ml_client.online_endpoints.invoke() with the name of the endpoint, the name of the deployment, and the json to be sent.

Question 19

What do you need to create a deployment on the endpoint?

Answer 19

Use ManagedOnlineDeployment with a name, the name of the endpoint, the model, the compute instance, and the number of instances

Question 20

What functions do you need to create a scoring script?

Answer 20

run() and init()

Question 21

What is special about a batch endpoint?

Answer 21

You can trigger it from Azure Synapse analytics or Azure Databricks. You can also integrate it with an existing pipeline

Question 22

What class is used to create a batch endpoint?

Answer 22

BatchEndpoint(name, description)

Question 23

How do you deploy an MLflow model to a batch endpoint?

Answer 23

Use BatchDeployment with extra specification parameters such as instance_count, max_concurrency_per_instance, mini_batch_size, output_action, output_file_name

Question 24

How do you invoke a batch endpoint?

Answer 24

You need an input() specifiying the path and asset type, then use ml_client.batch_endpoints.invoke()