Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Machine Learning > 4. Choosing the Right ML Infrastructure > Flashcards

4. Choosing the Right ML Infrastructure Flashcards

1
Q

ML models for video and images require a large number of computations for each loop of training. What mathematical calculations are involved?

A

matrix multiplications, additions, subtractions, and differentials

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

When you have an ML problem to solve, say image classification, you have three ways approaching it:

A

Pretrained (fastest to develop and least expertise needed)
AutoML (in between)
Custom (Most flexible but expertise needed)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Pretrained models are already deployed and can be readily used via …

A

APIs

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What is the biggest advantage of using pretrained models?

A

ease of use and the speed

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

How can developers use pre-trained models?

A

Using CLI, Python, Java, or Node.js SDK.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Are pre-trained models serverless?

A

Yes

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What is the biggest disadvantage of using pretrained models?

A

Less customizable

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What can Vertex AI AutoML do for you?

A

Build your own model using your own data.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

AutoML chooses the best ML algorithm, and the only thing that it needs is the data. What do you need to do?

A

Format the data and work on quality control

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Unlike with pretrained models, you have to provision cloud resources for training and deploying the model on instances. What do you have to decide on?

A

Number of hours of instance time
Devices the models need to deploy on (cloud, phone, IOT)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

What options do you have if pre-trained models and AutoML do not fit your need?

A

Use custom models in Vertex AI

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What pre-trained models does Google have?

A

Vision AI
Video AI
Natural Language AI
Translation AI
Speech‐to‐Text
Text‐to‐Speech

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What solutions does Google have in addition to pretrained models?

A

Document AI
Contact Center AI

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What can Vision AI do for you?

A

perform image classification, detect objects and faces, and read handwriting (through optical character recognition)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What is the process for Vision AI to image classification?

A

Detect objects in the photo
Get a set of labels for your image (e.g., table, plant, chair)
Get the dominant colors for these images
Categorize (e.g., Adult, Spoof, Medical, Violence)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What can Video AI do?

A

Recognize objects, places, and actions in videos.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Give 3 use cases for Video AI?

A

Build a video recommendation system
Create an index of your video archives
Map advertisements to your content

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What does Natural Language AI do?

A

Provides insights from unstructured text using pretrained machine learning models including entity extraction, sentiment analysis, syntax analysis, and general categorization.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

What does entity extraction do?

A

Identifies entities such as the names of people, organizations, products, events, locations, and so on.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What does sentiment analysis do?

A

Provides you a positive, negative, or neutral score with magnitude for each sentence, for each entity, and the whole text.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

What does syntax analysis do?

A

Identify the part of speech, dependency between words, lemma, and the morphology of text.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

Give 2 use cases for Natural Language AI

A

Measure the customer sentiment toward a particular product.
Use Healthcare Natural Language API to understand details specific to healthcare text like clinical notes or healthcare research documents.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

Translation AI has 2 levels, Basic and Advanced. What are the main differences?

A

Advanced version can use a glossary (a dictionary of terms mapped from source language to target language) and also can translate entire documents (PDFs, DOCs, etc.).

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

What do Media Translation API do?

A

Translates audio in source language into audio in target languages.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

What is the use case for Speech‐to‐Text service to convert recorded audio or streaming audio into text?

A

Creating subtitles for video recordings and streaming video as well.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
26
Q

There is AutoML training available for many data types and use cases. We can broadly categorize them into four categories. What are the categories?

A

Structured data
Images/video
Natural language
Recommendations AI/Retail AI

27
Q

What are the two methods of training models for AutoML for Tables?

A

BigQuery ML and Vertex AI Tables

28
Q

How do you deploy ML models trained by Vertex AI Tables?

A

Deploy the model on an endpoint and serve predictions through REST API.

29
Q

What metrics do you use for ML classification?

A

AUC ROC, AUC ROC, Logloss, Precision at Recall, Recall at Precision

30
Q

What metrics do you use for ML regression?

A

RMSE, RMSLE, MAE

31
Q

What metrics do you use for ML time-series forecasting?

A

RMSE, RMSLE, MAPE, Quantile loss

32
Q

Why do prices for node hours differ for the different types of AutoML jobs?

A

Because the hardware used for different AutoML jobs is different.

33
Q

Vertex AI offers three model training methods for forecasting. What are they?

A

AutoML
Seq2seq+ (takes in a sequence and produces another sequence)
Temporal Fusion Transformer (a deep neural network model that also uses the attention mechanism)

34
Q

Summarize all the available AutoML algorithms

A

Image classification (single): Predict one correct label from a list of labels
Image multiclass classification: Predict all the correct labels
Image object detection: Predict all the locations of objects
Image segmentation: Predict per‐pixel areas of an image with a label.
Video classification: Get label predictions for entire videos, shots, and frames.
Video action recognition: Identify the action moments in video.
Video object tracking: Get labels, tracks, and time stamps for objects

35
Q

What are the considerations for deploying AutoML models to edge devices (iPhones, Android phones and Edge TPU devices?

A

Less memory and low latency

36
Q

What are the four popular models for AutoML for Text?

A

Text classification: Predict the one correct label
Multi-label classification: Predict all the correct labels
Entity extraction: Identify entities within your text items.
Translation: Convert text from source language to target language.

37
Q

How to use Retail AI?

A

Upload the product catalog (product, photos, and other metadata)
Define “user events” (what the customer clicks, views, and buys)
Recommendations AI uses this data to create models for giving recommendations.

38
Q

What does Vision API Product Search do?

A

Trained on reference images of products in your catalog, which can then be searched using an image.

39
Q

Summarize the recommendation types available in Retail AI

A

Others you may like (product page, customer behavior and product relevance, click-through rate)
Frequently bought together (checkout page, shopping cart items, revenue per order)
Recommended for you (home page, user viewing history, click-through rate)
Similar items (product page, product catalog, click-through rate)

40
Q

Document AI has two important concepts: processors and Document AI Warehouse. What are they used for?

A

Document AI processor is an interface and do general processing, specialized processing (procurement, identity, lending, and contract documents), and custom processing (provide your own labeled set of documents).
Document AI Warehouse is a platform to store, search, organize, govern, and analyze documents along with their structured metadata.

41
Q

What can Document AI do?

A

Detect document quality
Deskew
Extract text and layout information
Identify and extract key/value pairs
Extract and normalize entities
Split and classify documents
Review documents (human in the loop)
Store, search, and organize documents (Document AI Warehouse)

42
Q

What does Dialogflow do?

A

Dialogflow is a conversational AI offering from Google Cloud that provides chatbots and voicebots.

43
Q

What do Agent Assist do?

A

Agent Assist can provide support by identifying intent and providing ready‐to‐send responses and answers from a centralized knowledge base as well as transcript calls in real time.

44
Q

What does Insights do?

A

Use natural language processing to call drivers (invoke driver software)
Measure sentiment to help leadership understand the call center operations

45
Q

What does Contact Center AI do?

A

Support multichannel communications between customers and agents.

46
Q

Why is GPU faster than CPU?

A

A GPU loads a block of memory and applies some operation using the thousands of ALUs in parallel, thereby making it faster.

47
Q

What machine series can use GPUs?

A

A2 and N1 Machine series

48
Q

Which one is fastest GPU?
NVIDIA_TESLA_T4NVIDIA_TESLA_K80
NVIDIA_TESLA_P4
NVIDIA_TESLA_P100
NVIDIA_TESLA_V100
NVIDIA_TESLA_A100

A

NVIDIA_TESLA_A100

49
Q

Where do you specify the type of GPU and its number?

A

Type of GPU: machineSpec.acceleratorType field in WorkerPoolSpec
The number of GPUs: machineSpec.acceleratorCount field in VM (worker pool).

50
Q

What are the restrictions for using GPUs?

A

Not all types of GPUs are available in all regions.
Use two or four NVIDIA TESLA_T4 GPUs on a VM but not three.
The GPU configuration must have sufficient virtual CPUs and memory compared to the machine type that goes with it.

51
Q

What is the characteristic of TPU?

A

Each TPU has multiple matrix multiply units (MXUs). Each MXU has 128 × 128 multiply/accumulators. Each MXU is capable of performing 16,000 multiply‐accumulate operations in each cycle using the bfloat16 number format.

52
Q

Cloud TPU provides the following TPU configurations:

A

A single TPU device
A TPU Pod (a group of TPU devices connected by high‐speed interconnects)
A TPU slice (a subdivision of a TPU Pod)
A TPU VM

53
Q

When to use CPUs?

A

Rapid prototyping that needs flexibility
Models that train fast
Small models that work with small batch size
Custom TensorFlow operations written in C++
Limited by available I/O or the networking bandwidth of the host

54
Q

When to Use GPUs?

A

Models for which source code does not exist or is too tedious to change
Models with a significant number of custom TensorFlow operations so they need to run at least partially on a CPU
Models with TensorFlow ops that are not available on TPUs
Medium‐to‐large models with medium‐sized batch

55
Q

When to Use TPUs?

A

Models that have a majority of matrix computations
Models that have no custom TensorFlow operations
Models that train for weeks or months
Large and very large models with very large effective batch sizes

56
Q

Cloud TPUs are not suited to the following workloads:

A

Programs that require frequent branching (conditional) and dominated element‐wise by algebra.
Sparse data (data that has lot of zeros)
High precision is not well suited for TPUs.
Deep neural networks that contain custom TensorFlow operations written in C++, especially if the custom operations in the main training loop.

57
Q

What is the main bottleneck for TPU?

A

The main bottleneck when using TPUs is the data transfer between the Cloud TPU and host memory.

58
Q

Prediction can happen in two methods:

A

online (real-time) and batch (reasonable time).

59
Q

What are the 2 important considerations when provisioning for predictions?

A

Scaling behavior and machine type

60
Q

What are the three resources you need to consider for configuring a GPU node?

A

CPU, GPU and memory

61
Q

You have the option of using GPUs to accelerate predictions, but there are some restrictions. What are the restrictions?

A

TensorFlow SavedModel or custom container designed to take advantage of GPUs. Not for scikit‐learn or XGBoost models.
GPUs are not available in some regions.
Use only one type of GPU DeployedModel resource or BatchPredictionJob
Limited on the number of GPUs you can add depending on machine types

62
Q

How to find the ideal machine type for a custom prediction container?

A

Deploy that container as a docker container to a Compute Engine instance directly
Benchmark the instance by calling prediction calls until the instance hits 90+ percent CPU utilization.
Determine the queries per second (QPS) cost per hour of different machine types.

63
Q

What is edge TPU for?

A

The Google‐designed Edge TPU coprocessor accelerates ML inference on these edge devices. A single Edge TPU can perform 4 trillion operations per second (4 TOPS), on just 2 watts of power. This is sold under the brand name of Coral.ai.
Edge TPU is for running ML inferences on edge devices which usually have limited bandwidth and may operate offline.

64
Q

What is ML kit for?

A

You can train your ML model on Google Cloud (AutoML or a custom model), and deploy the model into your Android or iOS app. The prediction happens in the device (low response times and enable offline prediction).

Machine Learning (14 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions