Big Data Lecture 01 Introduction

Question 1

What is the main learning objective of the course?

Answer 1

Learn to query gigantic amounts of data even when it is a bit messy.

Question 2

How is Data Science similar to Physics?

Answer 2

It is epistemic science of artificial data, so it has the same relation as Physics has to Mathematics, but to Computer Science.

Question 3

What was the first human data transmitting manner and what was its problem, and how was it solved?

Answer 3

People would speak or sing, however, this would get distorted over time. Solved by writing.

Question 4

What was the first data storing format? What is its problem and how was it solved?

Answer 4

Clay tablet table, tables are the most natural form of storing data.<br></br><br></br>Problematic copying, this was solved by the printing press.

Question 5

How was data stored in computers in history?

Answer 5

• 1960s: file base systems
• 1970s: relational databases
• 2000s: NoSQL era

Question 6

Three Vs of Big Data

Answer 6

• Volume
• Variety
• Velocity
• (Veracity)

Question 7

Why do we store more data?

Answer 7

<ul><li>We can, storage is cheap.</li><li>It carries value.</li><li>Combined data is worth more than sum of its parts.</li><li>We need data totality, some sites only operate well if they have all the data.</li></ul>

Question 8

Name prefix for unit: 1 000 (3 zeros)

Answer 8

kilo (k)

Question 9

Name prefix for unit: 1 000 000 (6 zeros)

Answer 9

Mega (M)

Question 10

Name prefix for unit: 1 000 000 000 (9 zeros)

Answer 10

Giga (G)

Question 11

Name prefix for unit: 1 000 000 000 000 (12 zeros)

Answer 11

Tera (T)

Question 12

Name prefix for unit: 1 000 000 000 000 000 (15 zeros)

Answer 12

Peta (P)

Question 13

Name prefix for unit: 1 000 000 000 000 000 000 (18 zeros)

Answer 13

Exa (E)

Question 14

Name prefix for unit: 1 000 000 000 000 000 000 000 (21 zeros)

Answer 14

Zetta (Z)

Question 15

Name prefix for unit: 1 000 000 000 000 000 000 000 000 (24 zeros)

Answer 15

Yotta (Y)

Question 16

Name prefix for unit: 1 000 000 000 000 000 000 000 000 000 (27 zeros)

Answer 16

Ronna (R)

Question 17

Name prefix for unit: 1 000 000 000 000 000 000 000 000 000 000 (30 zeros)

Answer 17

Quetta (Q)

Question 18

What are examples of different data shapes?

Answer 18

<ul><li>Tables,</li><li>trees,</li><li>graphs,</li><li>cubes,</li><li>text (unstructured).</li></ul>

Question 19

What is capacity?

Answer 19

How much data we can store.

Question 20

What is throughput?

Answer 20

How fast we can transmit data.

Question 21

What is latency?

Answer 21

How long till we start receiving data.

Question 22

What is the progress made in capacity, throughput and latency in last 70 years? What does this mean?

Answer 22

<ul><li>Capacity 23 000 000 000x,</li><li>Throughput 20 800x,</li><li>Latency 144x.</li></ul>

This is a big problem, now we need to parallelize.

Question 23

What is Big Data?

Answer 23

Porfolio of technologies that we designed to <i>store, manage and analyze data</i> that is too large to fit on a single machine while accommodating for the issue of growing discrepancy beween capacity, throughput and latency.