Serving and scaling models Flashcards
What are the three major challenges in managing ML features, and how does Vertex AI Feature Store address them?
The three major challenges are:
Sharing and Reuse: Features are often duplicated across projects and teams.
Low-Latency Serving: Serving features in production with low latency is difficult.
Training-Serving Skew: Misalignment between training and serving feature values.
Vertex AI Feature Store addresses these by:
Providing a centralized repository for feature sharing and discovery.
Ensuring low-latency feature serving with optimized infrastructure.
Computing feature values once for both training and serving, mitigating skew.
What is the difference between batch and online serving in Vertex AI Feature Store? Provide use cases for each.
Batch Serving:
Fetches large volumes of data for offline tasks like model training or batch predictions.
Example: Preparing a dataset for retraining a demand forecasting model.
Online Serving:
Retrieves small batches of data with low latency for real-time predictions.
Example: Fetching user preferences for personalized recommendations.
Define the key components of the Vertex AI Feature Store data model and their roles.
1) Feature Store: The top-level container for all features and their values.
2) Entity Type: A collection of semantically related features (e.g., “Users” or “Movies”).
3) Entities: Specific instances of entity types (e.g., “user_01” or “movie_02”).
4) Features: Attributes of entities (e.g., “age,” “average_rating”).
5) Feature Values: Data describing the features for a specific entity, timestamped to capture temporal changes.
How does the Feature Store help prevent data leakage during model training?
Feature Store uses point-in-time lookups to fetch feature values as they existed when the labels were generated. This ensures:
Temporal Consistency: Feature values align with the prediction time.
No Future Data Leakage: Prevents using information unavailable during the original event.
What are the prerequisites for data ingestion into Vertex AI Feature Store?
Data must include:
Entity ID: Uniquely identifies entities (must be a STRING).
Timestamp: Indicates when feature values were generated (optional if all values are simultaneous).
Feature Columns: Values matching the feature schema.
Data must be in:
BigQuery tables.
Cloud Storage files in Avro or CSV format.
Column headers must be defined (e.g., Avro schema, BigQuery column names).
Explain the concept of training-serving skew and how Feature Store mitigates it.
Training-Serving Skew: Mismatch between feature values used during training and those used during serving.
Mitigation:
Compute feature values once and reuse them.
Centralized storage ensures consistent feature definitions.
Monitor and alert for data drift.
Describe the role of an entity view in the Feature Store. What is the difference in retrieving features for online or batch serving?
An entity view represents a subset of features and their values for a given entity type.
Online Serving: Retrieves specific features for real-time predictions.
Batch Serving: Combines features across multiple entity types for offline tasks.
What is feature ingestion, and what methods does Feature Store offer?
Feature ingestion is importing computed feature values into the Feature Store.
Methods:
Batch Ingestion: Bulk import from sources like BigQuery or Cloud Storage.
Stream Ingestion: Real-time feature updates for online use.
How does Feature Store enable feature monitoring, and what can be tracked?
Feature monitoring tracks:
Data Drift: Changes in feature distributions.
Ingestion Metrics: Volume, processing time, errors.
Serving Metrics: CPU utilization, latency.
Alerts can be set for anomalies to ensure data quality.
What is the importance of timestamps in Feature Store and how they are used?
Timestamps associate feature values with their generation time:
Enable point-in-time lookups.
Track historical changes in features.
Facilitate time-series modeling.
What are the data retention policies in Feature Store?
Feature Store retains feature values based on their timestamps. The retention limit depends on the timestamp column, not the ingestion time.
How can Feature Store improve collaboration in ML projects?
Centralizes features for reuse across teams.
Provides APIs for easy discovery and access.
Implements role-based permissions for governance.
How are feature values stored for batch and online serving?
Offline Store: Retains historical data for training and batch predictions.
Online Store: Holds the latest feature values for low-latency retrieval.
What steps are involved in creating a Feature Store?
Preprocess and clean data.
Define feature store, entity types, and features.
Ingest feature values using batch or streaming methods.
Enable monitoring for quality control.
Describe the relationship between an entity and its features in Feature Store.
Entities are instances of entity types, and features describe specific attributes of these entities. For example:
Entity Type: “Movies.”
Entity: “movie_01.”
Features: “average_rating,” “genres.”
What is the minimum dataset size required for Feature Store ingestion?
Feature Store requires a minimum of 1,000 rows for batch ingestion to ensure data quality and usability.
How does Feature Store support feature discovery and sharing?
APIs: Search and retrieve features easily.
Centralized Repository: Ensures shared access.
Versioning: Tracks feature evolution for collaboration.
What are the value types supported by Feature Store, and why is this flexibility important?
Supported types: Scalars, arrays, and tensors (e.g., STRING, Boolean array).
Importance: Handles diverse data formats across ML models and tasks.
How does Feature Store handle array data types?
Arrays must use formats like Avro or BigQuery. Null values are disallowed, but empty arrays are acceptable.
Provide an example of using Feature Store for a real-world prediction task.
Example: Predicting baby weight based on historical features like birth location and mother’s age:
Batch ingest historical data into Feature Store.
Serve predictions to a mobile app using online serving API.
Give a high level outline of the steps needed to build an end-to-end pipeline to predict user churn using XGBoost?
The lab focuses on building an end-to-end machine learning pipeline that involves:
1) Training an XGBoost classifier in BigQuery ML to predict user churn.
2) Evaluating and explaining the model using BigQuery ML Explainable AI.
3) Generating batch predictions directly in BigQuery.
4) Exporting the model to Vertex AI for online prediction.
5) Leveraging Vertex AI for scalable predictions and MLOps.
How does BigQuery ML eliminate common ML workflow inefficiencies?
BigQuery ML:
1) Enables training and inference using standard SQL queries, eliminating the need to move data to separate environments.
2) Reduces the complexity of ML pipelines with fewer lines of code.
3) Integrates seamlessly with BigQuery’s scalable data storage and querying capabilities.
What are the advantages of deploying BigQuery ML models to Vertex AI?
Scalable Online Predictions: Provides low-latency, real-time predictions.
Enhanced Monitoring: Utilizes Vertex AI’s MLOps tools for retraining and anomaly detection.
Integration with Applications: Enables direct integration with customer-facing UIs like dashboards.
Why is the use of Google Analytics 4 data significant?
The Google Analytics 4 dataset used in the lab provides real-world user data from the mobile application Flood It! to predict user churn. It allows ML engineers to train realistic models for business-driven use cases.
Outline the key steps in deploying a Vertex Notebook for this lab.
Navigate to Vertex AI → Workbench → User-Managed Notebooks.
Click Create New, and select the TensorFlow Enterprise 2.11 environment.
Specify region and zone settings, then click Create.
Open the JupyterLab interface after setup completes.
How is a lab repository cloned in JupyterLab?
Open a terminal in JupyterLab.
Run the command:
git clone https://github.com/GoogleCloudPlatform/training-data-analyst
Navigate to the training-data-analyst directory to access lab materials.
What are the strengths of the XGBoost algorithm for churn prediction in this lab?
XGBoost is selected for its:
Efficiency in handling tabular data.
Robustness to missing values and feature importance ranking.
Proven effectiveness in classification tasks like churn prediction.
What is the purpose of Explainable AI in BigQuery ML?
Explainable AI provides feature attributions to:
Highlight the impact of individual features on predictions.
Build trust in the model by explaining its decisions.
Enable actionable insights for stakeholders.
How does BigQuery ML perform hyperparameter tuning for XGBoost models? and how is model performance optimised?
BigQuery ML allows users to:
Specify hyperparameter ranges in the training query.
Use automated searches to optimize performance.
Monitor tuning progress through query outputs.
What steps are involved in generating batch predictions in BigQuery ML?
Use the ML.PREDICT function with the trained model.
Specify input features for the batch dataset.
Write predictions to a new BigQuery table.
How is a BigQuery ML model exported to Vertex AI?
Export the model to a Google Cloud Storage bucket.
Specify the model name, format, and destination in the SQL query.
Use Vertex AI SDK to upload and deploy the model.
What tools and services must be enabled for batch prediction?
Services required:
BigQuery
AI Platform (Vertex AI)
Cloud Storage
IAM Credentials
Compute Engine
Use gcloud services enable to activate them.
What is the role of Cloud Shell?
Cloud Shell provides:
A persistent VM with pre-installed tools like gcloud and git.
Command-line access to Google
Cloud resources.
An authenticated environment linked to the project.
What are the key features of Vertex AI Prediction endpoints?
Real-Time Serving: Low-latency predictions for interactive applications.
Scalability: Automatically adjusts resources based on traffic.
Monitoring: Tracks performance metrics and model drift.
How is model evaluation performed in BigQuery ML?
Evaluation involves:
Running the ML.EVALUATE function on a test dataset.
Reviewing metrics such as accuracy, precision, recall, and AUC.
Using the results to fine-tune the model.
What are the benefits of using Vertex AI for online predictions compared to batch predictions in BigQuery ML?
Supports real-time applications like recommendation engines.
Scales dynamically for high traffic.
Provides integrated MLOps tools for monitoring and retraining.
What precautions should you take when using a temporary lab account?
Use Incognito mode to prevent conflicts with personal accounts.
Avoid using personal Google Cloud projects to prevent accidental charges.
Ensure all resources are deleted post-lab.
What data preprocessing steps are required before training the churn classifier?
Explore and clean the Google Analytics dataset.
Handle missing values
Normalize and encode categorical variables.
What SQL command is used to create a dataset in BigQuery for the lab?
CREATE SCHEMA dataset_name;
This initializes a dataset for storing training data and predictions.
Summarize the end-to-end workflow of training an XGBoost model in BigQuery.
Preprocess Google Analytics data in BigQuery.
Train and tune an XGBoost model with BigQuery ML.
Evaluate and explain the model using BigQuery tools.
Export the model to Cloud Storage.
Deploy the model to Vertex AI Prediction endpoints for online use.
What are the primary advantages of using ML pipelines, and why are they crucial for scalable machine learning workflows?
ML pipelines offer several key advantages:
Modularization: Each step in the ML workflow is encapsulated in its own container, allowing independent development and easier maintenance
Reproducibility: Pipelines track input and output from each step, ensuring consistent and repeatable machine learning processes
Scalability: They enable easier sharing of ML workflows across team members
Flexibility: Pipelines can be scheduled or triggered based on events like new training data availability
Dependency Management: Complex workflows with multiple interdependent steps can be managed more effectively
Tracking and Lineage: Provide comprehensive tracking of artifacts and how they’re used throughout the ML lifecycle
Key technical benefits include isolation of computational environments, version control of components, and the ability to implement conditional logic in workflow execution.
Explain the purpose and significance of the custom evaluation component in the Vertex Pipelines example, particularly its role in model deployment decision-making.
The custom evaluation component serves multiple critical functions:
Metric Retrieval: Extracts detailed evaluation metrics from the trained AutoML model
Visualization: Renders comprehensive metrics in the Vertex Pipelines UI, including:
ROC curve
Confusion matrix
Textual performance metrics
Deployment Decision Logic: Implements a programmatic approach to model deployment
Uses predefined threshold criteria (e.g., minimum area under ROC curve)
Provides a boolean decision on whether the model meets performance requirements
Enables automated, criteria-based model deployment
Supports reproducible and consistent model evaluation processes
The component demonstrates advanced ML engineering principles by incorporating automated quality gates into the ML workflow.
How do pre-built components in Google Cloud Pipeline Components enhance the ML pipeline development process?
Pre-built components provide significant advantages:
Abstraction of Complex Service Interactions: Simplify interactions with Vertex AI services
Standardized Integration: Provide consistent interfaces for common ML tasks
Reduced Boilerplate Code: Minimize the amount of custom code required
Support for Key ML Lifecycle Steps:
Dataset creation
Model training
Model evaluation
Model deployment
Enable Rapid Prototype Development: Accelerate pipeline construction
Ensure Best Practices: Incorporate Google Cloud’s recommended approaches to ML workflow management
Specific examples from the document include:
TabularDatasetCreateOp for dataset initialization
AutoMLTabularTrainingJobRunOp for model training
ModelDeployOp for model endpoint deployment
Describe the role of conditional logic in Vertex Pipelines and provide a concrete example of its implementation.
Conditional logic in Vertex Pipelines allows dynamic workflow execution based on runtime conditions. Key characteristics include:
Enables branching workflow execution
Supports complex decision-making processes within pipelines
Allows dynamic skipping or execution of pipeline steps
Concrete Example from the Document:
with dsl.Condition(
model_eval_task.outputs[“dep_decision”] == “true”,
name=”deploy_decision”
):
deploy_op = gcc_aip.ModelDeployOp(
model=training_op.outputs[“model”],
project=project,
machine_type=”e2-standard-4”
)
In this example:
The deployment step only executes if the model meets predefined performance thresholds
Uses the output of a custom evaluation component to make the deployment decision
Prevents deploying underperforming models automatically
Demonstrates advanced workflow control in machine learning pipelines
What are the key considerations when defining column transformations in an AutoML Tabular Training job?
Column transformations are critical for preparing data in AutoML Tabular Training:
Purpose: Define how different columns should be interpreted and processed
Types of Transformations:
Numeric: Standardize numerical features
Categorical: Prepare categorical variables for model consumption
Key Considerations:
Correctly identifying feature types
Ensuring appropriate feature representation
Handling potential data quality issues
column_transformations=[
{“numeric”: {“column_name”: “Area”}},
{“numeric”: {“column_name”: “Perimeter”}},
{“categorical”: {“column_name”: “Class”}}
]
Important aspects:
Explicitly define each column’s data type
Include all relevant features
Specify the target column for classification/prediction
Ensure comprehensive feature coverage
What functionality does Vertex Pipelines have available that supports MLOps principles of reproducibility and tracking?
Vertex Pipelines supports MLOps through several key mechanisms:
Artifact Lineage Tracking:
Tracks the creation and usage of artifacts
Provides visibility into how data and models are transformed
Enables comprehensive audit trails
Versioning:
Each pipeline run is versioned
Artifacts are tracked across different executions
Supports reproducibility of machine learning experiments
Metadata Management:
Captures detailed information about pipeline runs
Allows comparison of different model training attempts
Supports performance analysis and model iteration
Key Features Demonstrated:
Detailed artifact visualization
Lineage graph generation
Metrics comparison across pipeline runs
Comprehensive logging of model training processes
Explain the significance of the base_image parameter when creating custom pipeline components.
The base_image parameter is crucial for creating reproducible and isolated pipeline components:
Defines the container environment for component execution
Ensures consistent computational environment
Allows specifying:
Specific Python versions
Required system libraries
Pre-installed dependencies
Supports dependency management
Enables environment isolation between different pipeline steps
Example from the document:
@component(
base_image=”python:3.9”,
packages_to_install=[“emoji”]
)
Key Considerations:
Choose images that match project requirements
Consider performance and compatibility
Minimize image size for faster pipeline execution
Ensure consistency across different pipeline runs
What are the primary features and benefits of the Kubeflow Pipelines SDK in machine learning workflow management?
The Kubeflow Pipelines SDK offers comprehensive workflow management capabilities:
Workflow Composition:
Define complex ML pipelines as code
Create modular, reusable pipeline components
Support intricate computational graphs
Orchestration:
Manage dependencies between pipeline steps
Handle resource allocation
Support distributed computing scenarios
Extensibility:
Create custom components
Integrate with various ML frameworks
Support multiple deployment environments
Key Features:
Component decoration (@component)
Artifact management
Conditional execution support
Comprehensive SDK for pipeline development
Discuss the importance of defining input and output types in custom pipeline components.
Type definitions in pipeline components are crucial for:
Static Type Checking:
Validate component interfaces
Ensure type compatibility between pipeline steps
Catch potential errors during pipeline compilation
Semantic Understanding:
Provide clear contract for component inputs/outputs
Enable automatic documentation generation
Support tooling and IDE integration
Example from the document:
def emoji(text: str) -> NamedTuple(
“Outputs”,
[
(“emoji_text”, str),
(“emoji”, str),
]
):
# Component implementation
Benefits:
Improved code readability
Enhanced pipeline debugging
Better documentation
Supports complex return types using NamedTuple
How does Vertex Pipelines support scheduling and event-driven pipeline execution?
Vertex Pipelines enables advanced execution models:
Cloud Scheduler Integration:
Schedule pipeline runs at specific intervals
Support cron-like scheduling mechanisms
Event-Driven Triggers:
Execute pipelines based on specific cloud events
Respond to data updates
Support continuous model retraining
Key Capabilities:
Automated model retraining
Data drift detection
Periodic performance monitoring
Seamless integration with Google Cloud ecosystem
Scheduling allows for:
Continuous model improvement
Automated ML lifecycle management
Reduced manual intervention
