12_Vertex AI

Question 1

Scalar, Vector, Matrix, Tensor

Tensor = p-dimensional matrices

Question 2

Neural Networks

• Neural Network: model composed of layers, consisting of connected units (neurons)
• Neurons: Node, combines input values and creates one output value
• Input: what you need into a neuron (e.g. cat pic)
• Feature: input variable used to make predictions
  
  • Detecting email spam (subject, key words, sender address)
  • Identify animals (ears, eyes, colors, shapes)
• Hidden layer: set of neurons operating from same input set
• Feature Engineering: deciding which features to use in a model
• Epoch: single pass through training dataset
  
  • Speed up training by training on a subset of data vs all data

Question 3

Neuron

• Weights: multiplication of input values
• Bias: value of output given a weight of 0
• f: Activation Function

Question 4

AI Platform overview

• Fully managed Tensorflow (and other ML libraries) platform
• Distributed training and prediction:
  
  • Breaks jobs down into pieces, distributes to multiple workers
• Scales to tens of CPUs/GPUs/TPUs
• Hyperparameter tuning with Hypertune
• Automate the “annoying bits” of machine learning
• “I want to train my own model, but automate it.”

Question 5

Tensorflow

• Tensorflow is an open-source high-performance library for numerical computation that uses directed graphs
• Software library for high performance numerical computation
• Released as open source by Google in 2015
• Compatible with wide range of hardware and devices
• Often the default ML library of choice
• Pre-processing, feature creation, model training
• “I want to work with all the detailed pieces”

Question 6

How AI Platform works

• Prepare trainer and data for the cloud
  
  • Write training application in Tensorflow (or other ML library)
  • Python is language of choice
• Train your model with AI Platform
  
  • Master: manages other nodes
  • Workers: work on portion of training job
  • Parameter servers: coordinates shared model states between workers

Question 7

Get Predictions - Two types

• Online
  
  • High rate of requests with minimal latency
  • Give job data in JSON request string, predictions returned in its response message
  • In online learning, you train the system incrementally by feeding it data instances sequentially, either individually or by small groups called mini-batches. Each learning step is fast and cheap, so the system can learn about new data on the fly, as its arrives.
• Batch
  
  • Get inference (predictions) on large collections of data with minimal job duration
  • Input and output in Cloud Storage
  • In batch learning, the system is incapable of learning incrementally: it must be trained using all the available data. First the system is trained, and then it is launched into production and runs without learning anymore.

Question 8

Key Terminology

• Model: Logical container of individual solutions to a problem
  
  • Can deploy multiple versions
  • E.g. Sale price of houses given data on previous sales
• Version: Instance of model
  
  • E.g. version 1/2/3 of how the predict above sale prices
• Job: interactions with AI Platform
  
  • Train models
    
    • Command: ‘submit job train model’ on AI Platform
  • Deploy trained models
    
    • Command: ‘submit job deploy trained model’ on AI Platform
  • Failed jobs can be monitored for troubleshooting

Question 9

AI Platform process

Question 10

IAM

• Project and Models
  
  • Admin: Full control
  • Developer: Create training/prediction jobs, models/versions and send prediction requests
  • Viewer: Read-only access to above
• Models only
  
  • Model Owner
    
    • Full access to model and versions
  • Model User
    
    • Read models and use for prediction
    • Easy to share specific models

Question 11

Using BigQuery for data source

• Can read directly from BigQuery via training application
• Recommend to pre-process into Cloud Storage
• Using gcloud commands, only works with Cloud Storage

Question 12

Machine Scale Tiers and Pricing

• BASIC: single worker instance
• STANDARD_1: 1 master, 4 workers, 3 parameters servers
• PREMIUM_1: 1 master, 19 workers, 11 parameters servers
• BASIC_GPU: 1 worker with GPU
• CUSTOM: configure following parameters:
  
  • Machine type of master server​
  • ​# workers
    
    • If you specify one or more workers, you must also specify the type of machine to use for your worker nodes
  • # parameter servers
    
    • If you specify one or more parameter servers, you must also specify the type of machine to use for your parameter servers.
  • # evaluators
    
    • If you specify one or more evaluators, you must also specify the type of machine to use for your evaluators

GPU/TPU: Much faster processing performance

Pricing

• Priced per hour
• Higher cost for TPU/GPU’s

Question 13

Tensor Processing Unit (TPU)

Hardware processing specifically designed for machine learning

• Like a GPU, but even more optimized for ML
• Faster and more efficient

Question 14

Linear learner

The linear learner built-in algorithm is used for logistic regression, binary classification, and multiclass classification. AI Platform Training uses an implementation based on a TensorFlow Estimator.

A linear learner model assigns one weight to each input feature and sums the weights to predict a numerical target value. For logistic regression, this value is converted into a value between 0 and 1. This simple type of model is easy to interpret, because you can compare the feature weights to determine which input features have significant impacts on your predictions.

Question 15

Wide & Deep

The wide and deep built-in algorithm is used for large-scale classification and regression problems, such as recommender systems, search, and ranking problems. AI Platform Training uses an implementation based on a TensorFlow Estimator.

• Wide model is used for memorization.
• Deep model is used for generalization.
• A good use for the wide and deep model is a recommender system.

Question 16

Big Picture