Data lifecycle Flashcards
Four phases Data lifecycle steps
** Ingest
** Store
** Process/Analyze
** Explore/Visuzlize
Ways to ingest Data?
** Streaming
** Batch
** application
batch Streaming service
pubsub
Ingest batch
** cloud storage - Any binary object
** Storage transfer service
** BQ transfer service
** Storage transfer appliance PetaBytes
Batch application ingestion
** Cloud logging
** Cloud Pub/Sub
** CloudSQL
** Cloud Firestore
Cloud BigTable
** Cloud Spanner
Data lifecycle storing data
** cloud Storage
** Cloud storage for firebase
Databases
** Cloud SQL
** Cloud Spanner Structured
** BigTable - NoSQL
** Cloud Firestore - next gen cloud datastore, NoSQL
Warehouse
** BigQuery - serverless high
Blob
binary large object
Data lifecylce Process
** Compute Engine other services run on GCE
** GKE containerized - scale as needed More weight than GCE
Cloud Run next gen AppEngine
Processing data large-scale services
** Cloud Dataproc- data lake moder, etl, secure DS and scale
Hadoop, spark, flink, presto
** Cloud Dataflow - apache beam ground-up fashion
** Cloud Dataprep - intelligent
Data lifecycle Analyzing Services
BigQuery PetaBytes of data
Data lifecycle Exploring Services
** Cloud Datalab jupyet notebook
**
Data lifecycle visualizing services?
Services used to visualize data in GCP
** BigQuery BI
** Cloud Data Studio
** Looker