Storage and Protection Flashcards
Micro-partitioning size
Micro-partitions are usually broken into 50-500mb each (prior to compression).
Metadata on micro-partitions
- The range of values for columns in the micro-partition
- The number of distinct values
- Properties used for optimization and efficient query processing
Common Metadata Fields
- Object Definitions
- Object Properties
Object Definition Metadata
- Policies
- External Functions
- View definitions
Object Properties Metadata
- Object Name
- Object Column
How are metadata fields usually populated?
Using CREATE and ALTER statements for a given object.
Stage Types in Snowflake
- External Stages
- User stages
- Table stages
- Internal named stages
External Stages
Storage locations outside of Snowflake in another cloud storage location such as AWS/Azure/GCP. This allows for great flexibility for potential web hosting and ELT solutions prior to accessing the data in Snowflake.
User stages
Personal storage locations assigned to each user. These stages are unique to the user and are internal to Snowflake.
Table stages
Storage locations held/associated with a table object. This is particularly useful when staging and loading data exclusively meant for that table.
Internal Named Stage
Basically a custom defined stage internal to Snowflake within a database/schema.
File Format
An object within Snowflake that stores format information required to load data into a table from a stage.
File Format Benefits
Because it is an object, it can be re-used within other objects/statements like across several different COPY statements.
Snowflake Time Travel
Enables accessing historical data at any point within a defined period (90 days max).
Snowflake Time Travel benefits
- Restoring data-related objects
- Duplicating/backing up data from key points in the past
- Analyzing data usage/manipulation over specified periods of time
Data Retention (with Time Travel) periods by edition?
Standard Edition - 24 hours
Enterprise - 90 days
Data Retention Period for Transient and Temporary Objects?
Defaults to 24 hours, regardless of Snowflake edition
This can be altered though
Fail Safe
Separate from Time Travel, Fail-safe ensures historical data is protected in the event of a system failure or other event.
Fail-safe is a 7 day period during which any historical data is recoverable by Snowflake. This period immediately starts after the Time Travel retention period ends.
Is Fail-safe meant to be used as a back-up solution
No, it is a solution meant for recovery after a system failure or mistake.
Can a Snowflake user/customer recover their data themselves with Fail-safe?
No, Snowflake must do this for them.
Temporary/Transient Objects in Fail-safe?
Temporary and Transient objects are not recoverable with Fail safe.
What Encryption Method/Algo does Snowflake use?
AES-256 bit encryption with a hierarchical key model rooted in a hardware security model.
How often are keys rotated & rekeyed in Snowflake?
Keys are retired and rotated every 30 days, retired keys are then only used for decryption until they are destroyed 1 year after their creation. When retired keys are destroyed the data they protected is rekeyed with new keys.
