AI technology stack: Infrastructure

Question 1

What are the 4 main areas of the AI Technology Stack?

Answer 1

1) Compute
2) Storage
3) Network
4) Software

Question 2

What are the 2 main elements of Compute?

Answer 2

1) CPUs - Central Processing Units
2) GPUs - Graphical Processing Units

Question 3

What is the key to AI performance?

Answer 3

Match hardware to AI model requirements.

Question 4

What is the definition of “serverless”?

Answer 4

Not limited to a particular server or piece of hardware.

Question 5

What is the definition of “loose coupling”?

Answer 5

The ability to take data from a variety of sources - not limited to a particular network storage or a physical file share.

Question 6

Describe how AI is scalable.

Answer 6

It enables you to run multiple instances of code without being tied to a particular server.

Question 7

What is “high-performance compute”?

Answer 7

An isolated cluster of compute power. It includes high speed networking and specialized chip sets.

Question 8

What is “quantum computing”?

Answer 8

Data can be processed in 3 dimensions.

Question 9

What are “trusted execution environments”?

Answer 9

Humans are not in the loop.

Question 10

What are the 4 general stages of AI?

Answer 10

1) Ingestion
2) Preparation
3) Training
4) Output

Question 11

Name some storage considerations for AI.

Answer 11

• Expense of storage solution
• Different storage types (file, object, image, etc.)
• Structured vs. unstructured data (structured data is much easier to process)
• Flexibility (in order to do AI at scale)

Question 12

What aspects of networks are important to AI models?

Answer 12

• High speed network
• Understanding that different AI data types have different performance requirements. For example, an image file will take longer than a text file to get to the training algorithm.
• Protocols based on a congestion-free design.

Question 13

What is edge computing?

Answer 13

Edge computing is a distributed computing paradigm that brings computation and data storage closer to the sources of data, so that a user of a cloud application is likely to be physically closer to a server than if all servers were in one place. This is meant to make applications faster.

Example: The Internet of Things (IoT)

Question 14

Name some software implications for AI.

Answer 14

• Democratization of AI
• Tuning: Tuning an AI system allows you to customize models
• Transformation

Question 15

What are some tuning challenges?

Answer 15

• Number of hyperparameters
• Varies on model type and complexity
• Scaling trial and error tuning

Question 16

Why does data need to be transformed for an AI model?

Answer 16

For data compatibility.

Question 17

Methods of data transformation.

Answer 17

• Internal to the model
• External to the model (pre-processing)
• Output (post-processing)

Question 18

What is labeling?

Answer 18

Enriching data to use it for deployment, training and tuning.

Question 19

What are some considerations for labeling?

Answer 19

• Data labels must be of a high quality and standard.

Question 20

What are hyperparameters?

Answer 20

Parameters or values adjusted for an AI model or algorithm to tune it toward desired outcomes. The adjustment is mostly accomplished through trial and error.

Question 21

List some common hyperparameters.

Answer 21

• Learning rate
• Number of epochs (one cycle through the full training dataset)
• Momentum (the amount of history included in the equation)

Question 22

What is AI observability?

Answer 22

Monitoring the AI algorithm and metrics of the AI system.

Question 23

Why are Data and AI observability key to the success of any AI project?

Answer 23

They provide indices and metrics for performance, in-depth analysis of AI data and models, and the capability to investigate, resolve and prevent AI model issues.

Question 24

What are some challenges with AI observability and monitoring?

Answer 24

• Data integrity
• Data drift
• Bias and discrimination

Question 25

How can AI observability and monitoring challenges be mitigated?

Answer 25

Outcome validation

Question 26

What is a benefit of open source AI frameworks?

Answer 26

They give companies greater freedom to innovate with AI.