4.11 Big Data

Question 1

What is big data

Answer 1

a catch-all term for data
that won’t fit the usual containers

can be described in terms of volume, velocity or variety

volume - too big to fit into a single server
velocity - milliseconds to seconds to respond
variety - data in many forms such as
structured, unstructured, text, multimedia.

Question 2

What is fact based modelling in big data

Answer 2

Deconstructing big data into fundamental units known as “facts”

and placing these facts into a data set

Question 3

What can be said about the data set produced from fact based modelling

Answer 3

The dataset will be an ever growing list of immutable facts

Question 4

What are the advantages of fact based modelling (4)

Answer 4

No indexing needed
New items can simply be appended to the growing dataset
Facts are immutable making it easy to query
Data is true forever

Question 5

How can functional programming help when dealing with big data (3)

Answer 5

Statelessness
High order functions
Immutable data structures

all of which makes it easier to write correct
and efficient distributed code

Question 6

What is meant by statelessness in functional programming

Answer 6

Nothing about a functional program is dependent on how often it is called or what order other functions are called in

Makes it easier to write correct code and easy to understand

Question 7

What is meant by high order functions in functional programming

Answer 7

High order functions are functions which can take more than one input or output another function

High order functions are easily parallelised meaning more than one processor can work on different parts of the large data set at the same time.

This helps when having to deal with large volumes of data “Big data”

Question 8

What is meant by immutable data structures in functional programming

Answer 8

Functional programming languages are immutable - object cannot be modified once created.

Allows for easy parallel processing