Reliability Flashcards
Throttling
Responding to excessive API requests by dropping requests until rate falls below limit
3 parts of JSON Web Token (JWT)
Header
- Identifies token is a JWT
- Algorithm used to sign sign token
Payload
- Set of claims about issuer, subject, token
Signature
- Output of signature algorithm
- Shows signer knows secret and JWT token not altered
Storage Transfer Service
From HTTP/S, S3, Cloud Storage
To Cloud Storage
Transfer jobs run in GCP
*recommended for importing from other CSPs
Google Transfer Appliance
40TB
300TB
Cloud Storage
Time to transfer:
1 TB over 1 Gbps
3 hours
Time to transfer:
1 PB over 100 Gbps network
30 hours
Max volume and bandwidth for gsutil to transfer data
< 10 TB
At least 100 Mbps bandwidth
Data volume - when to use Transfer Appliance
> 20 TB
SLO
Agreed upon target for a measurable attribute of a service specified in a SLA
SLI
Metric that reflects how well a SLO is being met
Cloud Functions
Serverless service for running functions
Cloud Run
Serverless service for running stateless containers
Available as managed service or within Anthos
*Process up to 80 concurrent requests by default - up to 1,000
Compute Engine Machine Families (4)
- Optimized
- Key attribute
General purpose - best price performance
Compute Optimized - best performance per core
Memory Optimized - best memory per core (up to 12 TB)
Accelerator/GPU Optimized - parallel processing
Compute Engine persistent disk types (5)
*Strength
Standard persistent - efficient and reliable
Balanced persistent - fast, but cost effective (same IOPs as SSD persistent, but less per GB)
SSD persistent - single digit ms latency
Extreme persistent - for HPC in-memory databases, configurable IOPs
Local SSDs - Hot cache, flash optimized, low latency - higher higher throughput and lower latency than standard SSD
Instance Group Types
Managed
Unmanaged
Instance Template
- Definition
- What it contains
Used to create VM instances and MIGs
Machine Type Boot disk or container image Network settings Labels Startup scripts Other properties of VM
Cloud Functions events (5)
Cloud Storage Pub/Sub HTTP calls Firestore - document create, update, delete, write Cloud Logging
Cloud SQL Type Replication Failover Max size Scaling
MySQL, SQL Server, PostegreSQL
Read replica in same or different region
Google managed failover (primary to replica)
30 TB
Vertical scaling only - can enable auto scaling storage
Database Migration Service
Continuous replication for MySQL and PostgreSQL to Cloud SQL
Cloud Spanner Type Use Case Availability Query language Consistency Regions Scaling
Relational database for horizontal, global scaling
Apps that need strong consistency and global scale - Financial trading systems, logistics apps, global inventory tracking
99.999% availability
SQL and Google SQL
External consistency
Regional (3 copies in multiple zones)
Multi-region (3 copies in other regions)
Auto backup and replication
Auto sharding; increase CPU and RAM of instances to scale up
Bigtable Type Integration Use Cases Query language Locations Replications and failover Scaling
NoSQL, wide-column, PB scale, low latency
Apache HBase interface
Storing data for ML building
Streaming IoT data
Time series, marketing, financial, graph data
cbt
Data auto replicated between clusters
Scale clusters by adding nodes (tables are highly scalable)
Firestore
Type
Transactions
NoSQL, Document database for mobile, web, IoT apps
Supports ACID compliant transactions
Memorystore
Managed service for Redis and Memcached
Which IaaS can access Memorystore
Compute Engine
GKE
Cloud Functions
App Engine (requires Serverless VPC Access)
Firestore modes (2)
Datastore Mode - new server-based projects
Firestore Mode (native) - new web or mobile apps
Databases with time partitioned tables
BigQuery
Bigtable
Storage/database services that distribute data globally
Cloud Storage
Cloud Spanner
Cloud Firestore
VPC private IP ranges
- 0.0.0/8
- 0.0.0 - 10.255.255.255
- 16.0.0/12
- 16.0.0 - 172.31.255.255
- 168.0.0/16
- 168.255.255
BQ data batch loading options (4)
Load Jobs from Cloud Storage or local filesystems
BQ Data Transfer Service - loading from Google Ads, Ad Managers, Google Saas apps, 3rd party services
BQ Storage Write API
Firestore export
BQ data streaming options (2)
BQ Storage Write API
Dataflow job to write directly to BQ tables
BQ analysis pricing models (2) + free usage
- 1st TB free per month
On-demand - number of bytes processed each query
Flat rate (purchase slots/vCPU to run queries):
- Flex slots - commit to initial 60 sec
- Monthly - commit to initial 30 days
- Annual - commit to 365 days
BQ storage pricing models (2)
*First 10 GB free per month
Active storage
- table or table partitions modified in last 90 days
Long term storage
- table or table partition NOT modified for 90 consecutive days
Transfer Service for On-Premises Data
For moving billions of files and 100s of TB in single transfer
Agent based
Google Transfer Appliance (size options) (2)
40 TB or 300 TB
Cloud Logging export options (3)
- Where
- Format
Cloud Storage - JSON files
BQ - log tables
Pub/Sub - JSON messages
Use gsutil to transfer if less than __ TB and at least ___ bandwidth
10 TB
100 Mbps
Use Transfer Appliance if > than __
20TB
Workloads for GCP orchestration services (3)
Cloud Scheduler - schedule driven, single service
Workflows - Complex, microservices
Cloud Composer - data, ETL workloads
Helm
Package manager to configure and deploy K8 apps
GCP storage for ML learning data and artifacts
Cloud Storage
BQ
Cloud DLP classifies data in (3)
Cloud Storage
BQ
Datastore
Container Analysis
Auto scans container images in Artifact Registry and Container Registry for known vulnerabilities
Continuously analysis metadata for new vulnerabilities
Video Intelligence API
Annotate videos stored locally or in Cloud Storage, or live streamed with contextual info at per video, per segment, per shot, per frame levels
Recommended org wide quotas (3)
APIs per second
VMs created
Data queried per day in BQ
Recommendation Hub
UI that shows summary of all available recommendations in a project
Active Assist
Suite of tools to generate insights and recommendations to optimize cloud projects
Requester Pays
Storage configuration to bill department or team who requests data versus owner
Services with Committed Use Discounts. (spend based commitment) (5)
Cloud SQL Cloud Spanner Google Cloud VMWare Engine Cloud Run GKE