Path6.Mod2.a - Deploy and Consume Models - Batch Endpoint Deployment

Question 1

Same as Managed Endpoint Deployments…

W.r.t. Batch Endpoint Deployments, using this makes it easy and why it does…

Answer 1

Use MLFlow ; it automatically generates your Scoring Script and Environment.

Question 2

Like all usage of MLFlow models…

This specific action must happen to an MLFlow Model before it can be deployed to a Batch Endpoint

Answer 2

It has to be registered

Question 3

Just like MLTable…

This specific file must be included in the Model’s location for registration, before it can be deployed to a Batch Endpoint

Answer 3

To register, the folder location of the MLFlow Model file must also have the MLModel file (which describes how to load and use the Model).

Question 4

Create an instance of the Model class (required to register the Model in MFLow)

Answer 4

from azure.ai.ml.entities import Model
from azure.ai.ml.constants import AssetTypes

model_instance = Model(
name= 'mlflow-model',
		path='./model',
		type=AssetTypes.MLFLOW_MODEL)
model = ml_client.models.create_or_update(model_instance)

Question 5

ic mcpi mbs oa (SO AR) ofn rs (BRS)

BatchDeployment class: key behavior-controlling parameters

Answer 5

• instance_count: count of compute nodes to use
• max_concurrency_per_instance: max number of parallel Scoring Scripts runs per compute node
• mini_batch_size: number of files passed per Scoring Script run
• output_action: what to do with predictions:
  – BatchDeploymentOutputAction.SUMMARY_ONLY
  – BatchDeploymentOutputAction.APPEND_ROW
• output_file_name: file to append predictions to, if you selected BatchDeploymentOutputAction.APPEND_ROW above, otherwise ignored
• retry_settings: uses a BatchRetrySettings instance with params max_retries & timeout

Question 6

Near identical to ManagedOnlineDeployment

Create an instance of the BatchDeployment class, noting all the parameters

Answer 6

from azure.ai.ml.entities import BatchDeployment, BatchRetrySettings
from azure.ai.ml.constants import BatchDeploymentOutputAction

deployment = BatchDeployment(
    name="forecast-mlflow",
    description="A sales forecaster",
    endpoint_name=endpoint.name,
    model=model,
    compute="aml-cluster",
    instance_count=2,
    max_concurrency_per_instance=2,
    mini_batch_size=2,
    output_action=BatchDeploymentOutputAction.APPEND_ROW,
    output_file_name="predictions.csv",
    retry_settings=BatchRetrySettings(max_retries=3, timeout=300),
    logging_level="info",
)
ml_client.batch_deployments.begin_create_or_update(deployment)