Storage and Protection Flashcards
Micro-partitioning size
Micro-partitions are usually broken into 50-500mb each (prior to compression).
Metadata on micro-partitions
- The range of values for columns in the micro-partition
- The number of distinct values
- Properties used for optimization and efficient query processing
Common Metadata Fields
- Object Definitions
- Object Properties
Object Definition Metadata
- Policies
- External Functions
- View definitions
Object Properties Metadata
- Object Name
- Object Column
How are metadata fields usually populated?
Using CREATE and ALTER statements for a given object.
Stage Types in Snowflake
- External Stages
- User stages
- Table stages
- Internal named stages
External Stages
Storage locations outside of Snowflake in another cloud storage location such as AWS/Azure/GCP. This allows for great flexibility for potential web hosting and ELT solutions prior to accessing the data in Snowflake.
User stages
Personal storage locations assigned to each user. These stages are unique to the user and are internal to Snowflake.
Table stages
Storage locations held/associated with a table object. This is particularly useful when staging and loading data exclusively meant for that table.
Internal Named Stage
Basically a custom defined stage internal to Snowflake within a database/schema.
File Format
An object within Snowflake that stores format information required to load data into a table from a stage.
File Format Benefits
Because it is an object, it can be re-used within other objects/statements like across several different COPY statements.
Snowflake Time Travel
Enables accessing historical data at any point within a defined period (90 days max).
Snowflake Time Travel benefits
- Restoring data-related objects
- Duplicating/backing up data from key points in the past
- Analyzing data usage/manipulation over specified periods of time