Week 2 - Data Management

Question 1

What is Data management?

Answer 1

It includes the collection, storage, retrieval, quality assurance, and security

Question 2

explain data-information-knowledge-wisdom (DKIW)

Answer 2

Data: raw observations of the world
Information: data that has been processed to provide meaning
Knowledge: what makes possible the transformation of information into instructions or knowing how to do something
Wisdom: insight is integrated and actionable

Question 3

What is meta-data

Answer 3

Data that describes the properties or characteristics of end-user data and the context of those data

Question 4

What does meta-data do?

Answer 4

It enhances the searchability, categorisation and data management efficiency

Question 5

Structured data

Answer 5

Strictly organised such that it is easily searchable - database with a rigid schema

Question 6

Unstructured data

Answer 6

Requires special handling - email body, social media post

Question 7

Semi-structured data

Answer 7

Mix of both structured and unstructured data

Question 8

What is a database

Answer 8

An organised collection of logically related data

Question 9

Data management system

Answer 9

Data integrity: ensuring accuracy and consistency
Data security: protecting sensitive information
Scalability: adapting to growing amounts of data
Collaboration: enabling cross-functional access and analysis

Question 10

What is the fundamental database operations?

Answer 10

Create, read (retrieve), update, delete
It forms the basis of data manipulation and access

Question 11

What does ACID stand for?

Answer 11

Atomicity: all or nothing approach, smallest unit of transaction [buying concert ticket]
Consistency: ensuring that transactions bring the database from one valid state to another [library checkout]
Isolation: making sure transactions are processed independently [airplane seat tickets]
Durability: guarantees that once a transaction is committed, it will remain even in the case of a system failure [saving a paper]

Question 12

ACID are the principles that …

Answer 12

Ensure reliable transactions in a database

Question 13

Tabular data

Answer 13

[+] ideal for small amounts of data
[+] easy to create and use
[-] not suitable for complex relationships, only 2 dimensional

Question 14

CSV files

Answer 14

Text file often used for data exchange between different system

Question 15

What is a relational data-base

Answer 15

It is a collection of tables (relation) that interact with each other

Question 16

What does relational database enable

Answer 16

Enables complex queries and data manipulation

Question 17

What are its benefits?

Answer 17

Data integrity, flexibility, scalability, security

Question 18

What is the process of normalisation

Answer 18

Process of organising data in a database to reduce redundancy

Question 19

How do you normalise data?

Answer 19

Divide large tables into smaller, related tables and defining relationships between them

Question 20

What are the goals of normalisation

Answer 20

1. Improve data integrity and consistency
2. Optimise storage and query performance

Question 21

Columnar data base

Answer 21

Data stored in columns. Typically used for data warehousing

Question 22

What are the pros and cons of columnar databases?

Answer 22

[+] efficient data compression - data in column, same type
[+] queries that sum, count, average or otherwise aggregate values
[-] not suited for OLTP
[-] slower for write operations

Question 23

What are document databases?

Answer 23

No fixed schema. No SQL database designed to store, retrieve, and manage document -oriented information

Question 24

Schema Flexibility

Answer 24

Document databases typically allow for a flexible schema within the documents. Documents within the same collection may have different fields and structures

Question 25

Hierarchical Data representation

Answer 25

Documents can contain nested structures, arrays, and other complex data types, making them suitable for hierarchical data

Question 26

Distributed architecture

Answer 26

Distributed and can scale horizontally across multiple nodes or clusters

Question 27

Indexing and querying

Answer 27

Allowing for efficient search and retrieval of documents

Question 28

Lack of ACID transactions

Answer 28

Databases may not support full ACID properties across multiple documents or collections

Question 29

Graph Databases

Answer 29

Data entities represented as nodes and the relationship between them represented as edges

Question 30

Graph schema

Answer 30

Defines the types of nodes and relationships, while others are schema-less allowing for more flexibility

Question 31

Graph query language

Answer 31

Support GQL like Cypher to enable efficient querying and manipulation of graph structure

Question 32

Directed and undirected graphs

Answer 32

Directed - one way relationship
Undirected - two way relationship

Question 33

Graph databases are useful for

Answer 33

Interconnected relationship. Where the relationship is the defining characteristic of your data and your query is based on the relationship itself. The graph represents the model in a natural and intuitive way.

Question 34

Uses of graph databases?

Answer 34

Connections (linked in) knowledge discovery, recommender systems (TikTok)