Path7.Mod1.b - Responsible AI Dashboard - Creating your RAI Dashboard

Question 1

Con Too Gath Sco

Four steps to create a Responsible AI (RAI) Dashboard (Designer or Code)

Answer 1

Create a Pipeline using built-in Components. It should
1. Start with the RAI Insights Dashboard Constructor Component
2. Include one of the RA Tool Components
3. End with Gather RAI Insights Dashboard to collect insights into ONE dashboard
4. Optional: add the Gather RAI Insights Score Card at the end of the Pipeline

A complete pipeline looks like this:

The first three steps are REQUIRED

Question 2

Exp Cau Cf EA

Available Tool Components

Answer 2

• Add Explanation to RAI Insights Dashboard: Interpret models by generating explanations
• Add Causal to RAI Insights Dashboard: View Causal Effects of features on outcomes
• Add Counterfacturals to RAI Insights Dashboard: Explore how a change in input wouldchange the model’s output (What-Ifs)
• Add Error Analysis to RAI Insights Dashboard: Explore data distribution and identify erroneous subgroups

You can add any one or more of these into your dashboard build

These are in addition to the RAI Insights Dashboard Constructor, Gather RAI Insights Dashboard and Gather RAI Insights Score Card Components.

Question 3

Three tools for creating an RAI Dashboard

Answer 3

Just like anything else with Azure Machine Learning:
- CLI
- Python SDK (code experience)
- ML Studio Designer (no-code experience)

Question 4

MTLs RM Comps BP RP

Five general steps for using the Python SDK for building an RAI Dashboard

Answer 4

1. Register training and test Datasets as MLTables
2. Register MFLow-Scikit-Learn-flavor Model
3. Retrieve the desired built-in Components (for RAI)
4. Build the pipeline
5. Run the pipeline

Question 5

Three inputs for the RAI Insights Dashboard Constructor
- Inputs have to be in what formats
- Two restrictions on the first input

Answer 5

• The ML Model - currently only supports models in MLFlow format + sklearn Flavor!!!
• The training dataset - must be MLTable format
• The test dataset - must be MLTable format

Question 6

Python code for Step 3 “Retrieving desired RAI Components” when building your pipeline

Answer 6

First load the component:

This means getting the constructors for those components. Components require you to do this before you can use them (odd…).

Example:

// First load the component:
rai_constructor_component = ml_client_registry
    .components
.get(name="microsoft_azureml_rai_tabular_insight_constructor",
		label="latest")

// Then inside the pipeline:
construct_job = rai_constructor_component( 
    title="From Python",
		task_type="classification",
		model_input= model_input= Input(type=AssetTypes.MLFLOW_MODEL, path="<azureml:model_name:model_id>"),
		train_dataset=train_data,
		test_dataset=test_data,
		target_column_name=target_column_name,
		categorical_column_names='["location", "style", "job title", "OS", "Employer", "IDE", "Programming language"]',
		maximum_rows_for_test_dataset=5000,
		classes="[]", 
)

Question 7

m aml r t ?

The constructor identifiers for getting the Tool Component Constructors and what version they should be

Answer 7

They all have a similar pattern:
microsoft_azureml_rai_tabular_<what it is>
Note that we get the matching version of each follow up Component to that of the constructor (which is the “latest” version):

label = "latest"

rai_constructor_component = ml_client_registry.components.get(
    name="microsoft_azureml_rai_tabular_insight_constructor", label=label
)

// We get latest version and use the same version for all components
version = rai_constructor_component.version
print("The current version of RAI built-in components is: " + version)

rai_erroranalysis_component = ml_client_registry.components.get(
    name="microsoft_azureml_rai_tabular_erroranalysis", version=version
)

rai_explanation_component = ml_client_registry.components.get(
    name="microsoft_azureml_rai_tabular_explanation", version=version
)

rai_gather_component = ml_client_registry.components.get(
    name="microsoft_azureml_rai_tabular_insight_gather", version=version

Question 8

Python code fora full Gathering your Steps pipeline

Answer 8

from azure.ai.ml import Input, dsl
from azure.ai.ml.constants import AssetTypes

// Define as a Pipeline
@dsl.pipeline(
    compute="aml-cluster",
    experiment_name="Create RAI Dashboard",
)
def rai_decision_pipeline(
    target_column_name, train_data, test_data
):
    // Initiate the RAI Insights with its constructor
		rai_constructor_component = ml_client_registry.components.get(...)
    create_rai_job = rai_constructor_component(.....)
    create_rai_job.set_limits(timeout=30)

    // Add explanations
    rai_explanation_component = ml_client_registry.components.get(...)
    explanation_job = rai_explanation_component(
        rai_insights_dashboard=create_rai_job.outputs.rai_insights_dashboard,
        comment="add explanation", 
    )
    explanation_job.set_limits(timeout=10)

    // ***COMBINE EVERYTHING***
		rai_gather_component = ml_client_registry.components.get(...)
    rai_gather_job = rai_gather_component( constructor=create_rai_job.outputs.rai_insights_dashboard,
insight1=explanation_job.outputs.explanation,
// insight2=counterfactural_job.outputs.counterfactural, etc.
)
    rai_gather_job.set_limits(timeout=10)
    rai_gather_job.outputs.dashboard.mode = "upload"

    return {
        "dashboard": rai_gather_job.outputs.dashboard,
    }

Omitted full calls to ml_client_registry.components.get for brevity

Question 9

Two places to view or find your RAI Dashboard

Answer 9

1. Once the Dashboard job is done, on the Designer, click the View Button (top right corner)
2. The Responsible AI Tab of your Registered Model