Design and prepare a machine learning solution (20–25%)

Question 1

number of data formats

Answer 1

Three: Tabular/structured, Semi-structured, Unstructured

Question 2

Tabular/structured Data

Answer 2

Highly ordered data that has a schema. CSV, excel

Question 3

Semi Structured Data

Answer 3

Data is organized with key value pairs like a dictionary

Question 4

Unstructured Data

Answer 4

Follows no rules! Images, video, audio, documents

Question 5

What to do when your data format sucks?

Answer 5

Transform it to something more suitable

Question 6

Most common ways to store data for model training

Answer 6

Three: Azure Blob Storage, Azure Data Lake Storage, Azure SQL Database

Question 7

Azure Blob Storage

Answer 7

Can store structured and unstructured; cheapest option to store unstructured; most basic storage

Question 8

Azure Data Lake Storage G2

Answer 8

Stores CSVs as unstructured data, easy to give access for specific items to people due to hierarchical namespace, capacity is limitless

Question 9

Azure SQL Database

Answer 9

Only store structured data, like an sql table. ideal for data that doesn’t change over time

Question 10

What does Azure offer for compatible data formats

Answer 10

Azdb for Mysql, Azdb for Postgresql, Azdb for Mariadb, CosmosDB Cassandra API, CosmosDB MongoDB API

Question 11

Need Semi-structured data with on-demand schema

Answer 11

CosmosDB sql API

Question 12

Azure Synapse Analytics for pipelines

Answer 12

AKA Azure Synapse Pipelines. Can make data ingestion pipelines through UI or from json format. Makes it easy to ETL data from source into a data store

Question 13

Azure databricks for pipelines

Answer 13

Code-first tool where you can use sql, python, or R to define pipelines. Fast because it uses spark clusters

Question 14

Azure Machine Learning for Pipelines

Answer 14

Provides auto scaling compute clusters. Create ETL pipelines in Designer or from multiple scripts. Not as scaleable as Synapse or Databricks

Question 15

Six steps to train a model

Answer 15

1. Define problem 2. Get data 3. Prepare data 4. Train model 5. Deploy model 6. Monitor Model

Question 16

What questions to ask when defining the problem

Answer 16

Desired model output, what type of model is needed, what criteria make a model successful

Question 17

Five common ML tasks

Answer 17

Regression, Classification, Forecasting, Computer Vision, NLP

Question 18

4 services to train ML models

Answer 18

Azure ML studio, Azure Databricks, Azure Synapse Analytics, Azure Cognitive Services

Question 19

Azure ML studio for training

Answer 19

Can use UI, python SDK, or CLI to manage workloads

Question 20

Azure Databricks for training

Answer 20

Uses spark compute for efficient processing

Question 21

Azure Synapse Analytics

Answer 21

Primarily for ETL, but does have ML capabilities. Works well at scale with big data

Question 22

Azure Cognitive Service for training

Answer 22

Collection of prebuilt ML models for tasks like image recognition. Models can be customized through transfer learning

Question 23

How to save time and effort with pre-built models

Answer 23

Azure Cognitive Services

Question 24

Keep ETL and Data Science within same service

Answer 24

Azure Synapse Analytics or Azure Databricks

Question 25

Need full control of training and management

Answer 25

Azure ML or Azure Databricks

Question 26

Want to use Python SDK

Answer 26

Azure ML

Question 27

Want to use a UI

Answer 27

Azure ML

Question 28

CPU vs GPU

Answer 28

CPU is cheaper for smaller tasks, GPU is expensive for bigger tasks

Question 29

General purpose or Memory optimized

Answer 29

General is obvious, use memory optimized for larger datasets or notebooking

Question 30

Spark compute

Answer 30

Spark clusters are distributed so they work in parallel. can use gpu and cpu. Databricks and Azure Synapse Analytics use this

Question 31

Prediction options for endpoints

Answer 31

Real-time or batch

Question 32

Real-time predictions

Answer 32

Low latency, need the answer NOW! Good for customer facing services. Works on a single row input of data

Question 33

batch predictions

Answer 33

Score new data in batches, save results as file. Good for forecasting or scheduled predictions. Works on multiple rows of data

Question 34

Compute for real-time predictions

Answer 34

Container services like Azure Container Instance ACI or Azure Kubernetes Service AKS. Compute is always on and costing money

Question 35

Compute for batch predictions

Answer 35

Clusters offer scoring in parallel. Compute spins down when not actively working