Storage and Database Services Flashcards
Is your data (you want to store) structured?
If it is not you go to chose between Filestore or Cloud Storage
If it is, you need some sort of database: Cloud Spanner, Cloud SQL; Memory Store, Fire Store; BigTable, BigQuery (now you ask do your data involve analytics, is it relational, do you need to scale…
Storing unstructured data, do you need a shared file system?
If yes you need Filestore
if not you want Cloud Storage
For structured does your workload involve analytics?
if yes you need to go to (BigQuery data warehouse with sql querying/BigTable(no sql high throughput)
If not you need to ask if your data is relational
For Structured relational data (which doesn’t involve analytics)
You ask if you need global scalability
If yes you need Cloud Spanner(horizontal scaling and global availability)
if not you use Cloud Sql
For Structured un relational data (which doesn’t involve analytics)
If you need application cashing
you use Memorystore (in memory database, fully managed redis service)
otherwise you use Firestore (document database)
Cloud Storage
GC object storage service, www storage and retrieval of data at any time.
It is collection of buckets that you place objects into.
Scalable to exabytes; time to first byte in milliseconds, single api across storage classes and high availability across storage classes.
You can not easily index files; you need specific url to access objects
In Cloud Storage, objects are immutable, which means that an uploaded object cannot change throughout its storage lifetime.
you can change storage class but not location type
IAM can control which individual users or service account have different roles, rolles are inheried from prject to bucket to object
ACL (access control list)
signed URLS (expire after limited time)
Cloud Storage storage classes
t has 4 storage classes
1.Standard (best for frequently accessed data, and stored for brief periods, most expensive, no storage duration, no retrieval cost)
2.Nearline low cost highly durable for storing infrequently accessed data ??? once in month (30 days duration??)
3.Coldline 90 day minimum storage duration lower availability
4.Archive for disaster recovery, backups, mili seconds for retrieving data, min duration a year, no sla’s
Cloud Storage location types
- regional (optimize latency
and network bandwidth for data consumers, such as analytics pipelines that are grouped in the same region.) - dual-regional (similar performance advantages as region and high availability that comes with being geo-redundant)
- multiregional (you want to serve content to data consumers that are outside of the Google network,
and distributed across large geographic areas)
Buckets
Claud Storage is broken into Buckets global unique name, can’t be nested, gsutil command to access data (or json/xml apis)
Cloud Storage features
-use your own encryption keys
-automatically delete archived objects
-maintain multiple versions
-synchronise VM directory with a bucket
-object change notifications using PubSub
Lifecycle Management configuration examples
-downgrade the storage class of objects older than a year to Cloud line storage.
-delete objects created before a specific date, for example, January 1st, 2017.
-keep only the three most recent versions of each object, any bucket with versioning-enabled.
updates to your lifecycle configuration may take up to 24-hours to go into effect.
services that can upload terabytes, even petabytes of data to a bucket
3 services:
- Transfer Appliance (securely migrate large volumes of data from hundreds of terabytes up to one petabyte)
- Storage Transfer Service (high-performance imports of online data)
Offline Media Import (import is a third-party service, where physical media such as storage arrays, hard disk drives, tapes, and USB flash drives is sent to a provider who uploads the data.)
