Big Data

Question 1

Define

Big Data

Answer 1

A broad term for datasets so large or complex that traditional data processing applications are inadequate, and the data must be stored on multiple servers.

Question 2

Define

Volume

The Three V’s

Answer 2

The capacity required to store the data exceeds a single server.

Question 3

Define

Velocity

The Three V’s

Answer 3

The data is produced and/or processed at very high speed.

Question 4

Define

Variety

The Three V’s

Answer 4

The data is very diverse; data can appear in different types (eg text, video, images) and forms (eg structured, unstructured, semi-structured)

Question 5

Define

Structured Data

Answer 5

Data that can be stored in a traditional system such as a relational database or spreadsheet, as they can be defined using fields and records.

Question 6

Define

Unstructured Data

Answer 6

Data that cannot be defined in columns or rows (text documents, PDFs, voice messages, emails). It makes it difficult to anlayse the data.

Question 7

Identify

Issues With Big Data

Answer 7

• Data sets so large they are difficult to store and analyse.
• Data is constantly changing, so it is difficult to keep track of changes.
• Massive storage and processing power required.
• Specialised software required to manage and extract meaningful info from the data.
• Data is unstructured so makes it very difficult to analyse.

Question 8

Describe

Data Mining

Answer 8

The use of a variety of statistical analysis tools to uncover previously unknown patterns in the data stored in databases or relationships among variables.

Question 9

Describe

Predictive Analysis

Answer 9

The use of data warehouses and complex algorithms to forecast future events, based on historical trends and calculated probabilities.

Question 10

Describe

Data Warehousing

Answer 10

The process of bringing together data from various sources into one place so that meaningful data analysis can take place, such as data mining and predictive analytics.

Question 11

Describe

Fact-Based Model

Answer 11

Used to represent, model, and query data sets at the scale of Big Data. It is similar to entity relationship models used in databases.

Question 12

Define

Fact

Fact-Based Model

Answer 12

A piece of data that cannot be decomposed any further, and is forever true. The data:
- Must not include reduntant information.
- Must be specific to a particular point in time.
- Cannot be changed or deleted.

Question 13

Describe

Graph Schema

Answer 13

A method of defining a structure of a big dataset using the fact-based model, as a graph.

Question 14

Describe

Node

Answer 14

Represents a core entity in a data set. Depicted with an oval.

Question 15

Describe

Edge

Answer 15

Represents the relationships between entities (nodes). Depicted using solid lines linking nodes togethor.

Question 16

Define

Property

Answer 16

Defines information about a node. Depicted with a rectangular box.

Question 17

Define

Distributed Processing

Answer 17

The principle of dividing processing work between two or more computers, linked together in a network.

Question 18

Define

Functional Programming

Answer 18

A type of programming paradigm that is mainly used for calculations and distributed processing, as the code can be proven correct and can be distributed across multiple devices without fear of erroneous results Some characteristics are:
- Immutable Data Structures
- Statelessness
- Higher-Order Functions

Question 19

Define

Immutable Data Structure

Answer 19

A data structure in which one cannot insert, remove, or replace the values contained therein.

Question 20

Define

Statelessness

Answer 20

A given program does not change its state during execution (data structures don’t change and variables are not used).

Question 21

Define

Higher-Order Function

Answer 21

A function that can use functions as parameters and return functions as a result.

Question 22

Define

Map Function

Answer 22

A higher-order function which applies a function to each element of a list and returns a new list.

Question 23

Define

Fold/Reduce Function

Answer 23

Applies a function recursively over a list and returns a value.