Theory: Big Data

Question 1

Give the three defining features of big data

Answer 1

-volume
-velocity
-variety

Question 2

Describe the volume feature of big data

Answer 2

-too much data to fit in a conventional hard drive or on a server. Data is therefore stored across multiple servers, each composed of many hard drives.

Question 3

Describe the velocity feature of Big Data

Answer 3

Refers to the rapid modification of data. Servers must respond to frequently changing data

Question 4

Describe the variety component of big data

Answer 4

Big data consists of many different types of data

Question 5

What is the most challenging quality of big data?

Answer 5

It’s lack of structure

Question 6

Why does the unstructured nature of big data cause issues?

Answer 6

Makes data difficult to analyse

Question 7

How is the problem of lack of structure overcome?

Answer 7

Machine learning techniques are used to discern patterns in data

Question 8

What is a second problem with Big data?

Answer 8

As data is stored across multiple servers, the processing must also be across multiple machines. With conventional programming paradigms, all servers would have to be synchronised to prevent data being overwritten.

Question 9

How is data being stored across multiple servers dealt with?

Answer 9

Functional programming can be used to process data across multiple servers.

Question 10

What are some of the features of functional programming paradigms?

Answer 10

Functional programs are stateless (meaning there are no side effects) and make use of immutable data structures. It supports higher-order functions.