3.11 Big Data

Question 1

Big Data meaning

Answer 1

describes data whose volume is too large to fit on a single server and is generally unstructured

Question 2

three defining features of big data

Answer 2

volume, velocity, variety

Question 3

volume big data

Answer 3

there is too much data for it to fit on a conventional hard drive or server, stored over multiple servers composed of many hard drives

Question 4

velocity big data

Answer 4

data is created and modified rapidly within milliseconds

Question 5

variety big data

Answer 5

data held consists of many different types of data

Question 6

big data’s biggest challenge

Answer 6

lack of structure, makes it difficult to analyse data as databases are not suited to store it all, to extract useful info machine learning techniques must be used to find patterns in data, data is distributed across several servers

Question 7

functional programming and Big Data

Answer 7

solves problem of processing data over multiple machines, functional programs are stateless and make use of immutable data structures (state can not change after creation), makes it easy to write correct, efficient, distributed code

Question 8

fact-based model

Answer 8

each piece of information is stored as a fact, facts are immutable and can not be overwritten, stored with a timestamp to show the most recent, reduces risk of losing data accidentally, appends new data