Data Integration & Data Quality

Question 1

Which three data models are there?

Answer 1

1 - hierarchical model
2 - network model
3 - relational database model

Question 2

Hierarchical data model

Answer 2

Data is structured as a tree of records

Question 3

Network model

Answer 3

Allows nodes to have multiple parents, allowing for more flexibility

Question 4

What does the relational database model provide?

Answer 4

An independent way to store data. This is the only data model in which data can be restructured without affecting the applications.

Question 5

Approaches against physical location challenges

Answer 5

1 - data federation

2 - data warehousing

Question 6

Data federation

Answer 6

Leaving the data in its place and moving the query to the data. The database engine becomes the federation engine or the mediator, which maintains a global target schema to provide a virtual view of the integrated data. The query optimizer then uses this global target schema to decompose a query into partial queries that can be executed by each of the sources.

Question 7

What are wrappers used for in data federation?

Answer 7

They are used for each of the sources to map the global schema to the schema of the source and to negotiate how much of a partial query each of the sources can do and at what cost. It translates the query and returns to result back to the federation engine.

Question 8

What to do if the data is described differently in different places?

Answer 8

Map the existing schemas to one common schema. This is a virtual schema in the case of federation and a materialised schema in the case of data warehousing.

Question 9

What do declarative schema mappings describe?

Answer 9

The relationships between the schemas of heterogeneous data sources.

Question 10

Name the measurements of data quality

Answer 10

1 - accuracy
2 - timeliness (update frequency)
3 - completeness
4 - consistency
5 - duplication
6 - referential integrity (first insert, then updates, etc)
7 - domain integrity (ages should be in a certain domain)
8 - follow the business rules (if there can only be one manager, not multiple managers in the database)

Question 11

What are the key aspects of data distribution?

Answer 11

1 - availability
2 - scalability
3 - transparency (of access)
4 - reliability/fault-tolerance

Question 12

Homogenous distributed databases

Answer 12

When websites have identical software, are aware of each other, and agree to cooperate in processing user requests.
It appears as a single system to the user.
It is a closed world assumption, everything in the network is known.

Question 13

Heterogeneous distrbuted databases

Answer 13

Different sites may use different schemas and software. They might not be aware of each other and may provide only limited facilities for cooperation in transaction processing (queries)

Question 14

Horizontal fragmentation

Answer 14

Each tuple of the relation is assigned to one or more fragments
For example, storing the employees of each department in a separate database.

Question 15

Vertical fragmentation

Answer 15

The schema for the relation is split into several smaller schemas

Question 16

What does vertical fragmentation require?

Answer 16

All schemas to have a common candidate key / super key to ensure losless joins (unique id)

Question 17

Communication heterogeneity

Answer 17

Some databases might allow for direct query language while others offer APIs

Question 18

Schema heterogeneity

Answer 18

The structure of the table is different

Example: student_info table and student_classes and student_contact_info

Question 19

Data type heterogeneity

Answer 19

Data stored as different data types

Question 20

Value heterogeneity

Answer 20

The same logical values being stored in different ways

Example: str, st, street

Question 21

Semantic heterogneity

Answer 21

The same values meaning different things in different sources
Example: title -> job title or title of a person

Question 22

When are federated databases useful?

Answer 22

When there are many sources and only a few are communicating.

Question 23

Name the two approaches to update data warehouses

Answer 23

1 - complete rebuild

2 - incremental updates

Question 24

Name two important things about data warehouses

Answer 24

1 - A data warehouse is never up to date at all times

2 - They cannot handle streaming data

Question 25

Data lake

Answer 25

A data lake stores all types of structured and unstructured data as-is.
Can be combined with data warehouses

Question 26

Mediator

Answer 26

A virtual view over the data. It has a virtual schema that combines all schemas from the sources. The mapping takes place at query time. The mediator sends queries to the wrappers, which connect to the course and send back the result to the mediator. The mediator combines all the results into one final result.