Path6.Mod2.a - Deploy and Consume Models - Batch Endpoints

Question 1

When to use Batch Endpoints

Answer 1

With long-running tasks that deal with large amounts of data performed through batch operations.

Question 2

Batch Inferencing

Answer 2

ML uses Batch Inferencing to asynchronously apply a predictive model to multiple cases (like async batch scoring), then persists the results to file or DB or some other datastore connected to your ML Workspace.

Question 3

BSJ CC m_i

Batch Endpoints
- When invoked creates this kind of Job
- The Job requires this kind of Compute
- For new data in parallel batches, the Compute has to have more than one

Answer 3

• When the Endpoint is invoked, it submits a Batch Scoring Job for execution
• Batch Scoring Jobs require a Compute Cluster for scoring multiple inputs
• The Compute Cluster requires a max_instances value greater than 1

Question 4

Create an instance of the BatchEndpoint class

Answer 4

Straightforward and pretty simple:

endpoint = BatchEndpoint(
    name="endpoint-example",
    description="A batch endpoint",
)

ml_client.batch_endpoints.begin_create_or_update(endpoint)

Question 5

The Batch Endpoint name does NOT have to be unique per Region, but must be unique per Workspace (T/F)

Answer 5

False. The name must always be unique per Region

Question 6

A BatchEndpoint instance must be created first, before Deployments are added to it (T/F)

Answer 6

True

Question 7

You can deploy multiple Models to a BatchEndpoint (T/F)

Answer 7

True

Question 8

The type of Deployment the Batch Scoring Job uses, unless otherwise specified and how to trigger the Deployment

Answer 8

It uses the default deployment. Triggered by hitting the batch endpoint.

Upon viewing the Scoring Job’s details, you’ll see under Deployment Summary the deployment details:

Question 9

Create an instance of an AmlComputer class for a Compute Cluster that will support Batch Endpoints

Answer 9

from azure.ai.ml.entities import AmlCompute

cpucluster = AmlCompute(
    name="aml-cluster",
    type="amlcompute",
    size="STANDARDDS11V2",
    mininstances=0,
    maxinstances=4,  // main thing is to specify more than one max instances
    idletimebeforescaledown=120,
    tier="Dedicated",
)

cpucluster = mlclient.compute.begin_create_or_update(cpucluster)

Question 10

The advantage of using a Compute Cluster for running the Scoring Script

Answer 10

You can run the Score Script on separate instances in parallel.