Final Review Sheet

Question 1

index

Answer 1

table only has two attributes, a key identifier attribute and a pointer attribute that tells you which block of data it is in

Question 2

B+tree index

Answer 2

it follows the pattern of a tree the pointers have a top level then they go to the next level and to the next level until you get to the lowest grain of data
used for items with a large cardinality, it is more geared towards transactional systems

Question 3

block

Answer 3

logical unit of data transferred between disk and memory is block (4 bytes)

Question 4

bitmap index

Answer 4

a collection of bit vectors where there are only 1’s and 0’s

it is used for boolean operations and in data warehouses

Question 5

Data Staging Steps

Answer 5

Extraction
Data cleansing
Data Integration
Transformation
Loading
Maintenance

Question 6

Extraction

Answer 6

take it from the source, ex: operational systems, flat files, web

Question 7

Data Staging

Answer 7

data is placed in a staging area away from the original system to cleanse it

Question 8

Data Cleansing:Missing Values

Answer 8

1. Exclude the record
2. Exclude the attribute/field
3. Replaced by a global constant
4. Replaced by the attribute mean
5. replaced by the most probable value
   6 Manual correction
6. Apply specific algorithm

Question 9

Data Integration

Answer 9

data from different sources with different data formats need to be integrated into one data warehouse

problems: same names between different attributes or same attribute with different names
1. Get source and target schemas
2. Integrate source schemas and map to data warehouse schema with the help from tools and business

Question 10

Transformation

Answer 10

Prepare data for loading into the data warehouse

1. change data formats
2. Create derived attributes and tables
3. Aggregate - make aggregate fact table
4. Create warehouse keys

Question 11

Loading

Answer 11

Load cleansed, integrated, transform data into data warehouse

Question 12

Data integration tools

Answer 12

Pentaho
Oracle
SQL
Commercial 
lower maintenance costs, easier to use, metadata is useful by default
Scripting/SQL
more productive, specific to your needs
more difficult to maintain

Question 13

DW Maintenance

Answer 13

DW refreshment, is the process of keeping data in DW consistent/current with data in source systems,
can be based on user driven policy where it is only refreshed when asked for
also it can be based on a warehouse driven policy that has a schedule of refreshing it

Question 14

Trends in Data Warehousing

Answer 14

Online forums are used to predict product defects example: honda
social networks are used to predict stock behavior example: stocktwitter
Marketing trends are predicted by who you are in contact with on your phone

Question 15

mapreduce

Answer 15

a programming model invented at google. uses thousands of commodity pcs connected by ethernet
map reads an input and produces a key value pair, all pairs are associated with the same key and are grouped then passed to reduce. multiple computers do these things
reduce: receive a group of pair and merge(aggregate) them

Question 16

hadoop

Answer 16

open source software for reliable, scalable, distributed computing

Question 17

cloud computing

Answer 17

data warehouse services including but not limited to ETL, data management, reporting, dashboards, etc. access control is offered via the internet

Question 18

OLAP

Answer 18

online analytical processing
places key performance indicators into context
measures are pre aggregated and precalculated
It is organized in the form of a cube and allows data to be modeled and viewed in multiple dimensions
Olap uses Role up(drill up) which summarizes data by climbing up a concept hierarchy or by reducing dimensions example: aggregating all of the cities in one state into one field
Also uses Drill-down which moves down a concept hierarchy and adding dimensions to be more specific
Typical operations include
slice:creates a slice from the cube by choosing a single value for one of the dimensions
dice: creates subcube from the cube by choosing two or more values for one or more of the dimensions