Chapter 4: BackRoom Flashcards
Backroom is also known as
Data acquisition
Back room is defined by
Business requirements
Source Systems
-Can be from various places
-can be from legacy systems, report or flat file
-dictates data storage types
Data Staging Area
-where data transformation takes place
-DW value is created
Data Staging Storage Types
-Flat files, relational tables and proprietary structures
-conformed dimensions are created
-Emergency recovery is stored here
Presentation Servers
-stores data for end user queries and reports
-data detail and aggregate data loaded by business segment
Data Staging System Requirement
-Productivity Support
-Usability
-Metadata driven
Productivity Support
need capabilities to support code library check in and check out, version control and prod & dev system builds
Usability
-Increase usability by improving system documentation
Metadata driven
Information about tables should be accessible
Extract Services
-pulling data from the source systems
-60% of development time is spent in the extract process
-Challenge what data to pull and filters to apply
-
Major Extract Requirements
- Multiple sources
-Code generation
-Multiple extract types
-replication
-compression/decompression
Multiple extract Types
-Incremental Loads
-Transaction Events
-Full Refresh
Replication
continually updating the tables during the day
Incremental Loads
Use date of the last load as metadata
Transaction events
use transaction log file to identify changed records
Full Refresh
Pulling entire tables instead of pulling solely the changes
Data transformation Services
Transforming the data into a presentable format for the end users
Transformation abilitites
-Cleansing, Deduping, Merging
-Data Type Conversion
Data Loading Services
-Function of the target platform
Asset Management : Back Up and Recovery
Process to facilitate the return of the DW in case of system failure
What does Back up system need?
-High performance
-Simple administration
-Automated Lights-out operations-
archive and retrieval
-Process allowing users to retrieve older data
-it reduces expenses by using inexpensive media
-Dictating how long data should be kept before archiving
-Better solution to archive data during load process
Backup and Archive Planning
- Start with a backup plan
- Determine an appropriate backup process
- Implement the process
- Practice