1 Data Science & Big Data

Question 1

Define Data Science

Answer 1

Two definitions:
- Processing, representation, value-extraction, and knowledge extraction of data (study of data)
- Data-Driven Science (a tool for scientific discovery in other scientific fields). It complements Hypothesis-Driven Science

Question 2

Hypothesis-Driven Science Process

Answer 2

1. Hypothesis
2. Perform Experiment
3. Check Outcome

Question 3

Data-Driven Science Process

Answer 3

1. Take existing data
2. Perform analysis
3. Observe results

Question 4

Physics Equations
Historical Books/Artifacts
Earth Data
Biomedical Data
User Activity
User Content

Answer 4

Sources of Data

Question 5

Is data often intended for scientific use?

Answer 5

No. Normally its original purpose is for an activity of different purpose. Eg: clinical practice, accounting, sharing knowledge, etc.

Question 6

Possible outcomes of a data science analysis

Answer 6

1. Useful insight (can be published)
2. Inform on further experiments to be conducted to verify a hypothesis
3. Inform whether it is feasible to build a system that accurately predicts the data

Question 7

What does it mean that data is high-dimensional

Answer 7

Data cannot be plotted easily in two dimensions

Question 8

How machine learning based dimensionality reduction works

Answer 8

The machine tries to recreate the original distances between points (x) from a high-dimensional space using points (y) in a lower-dimensional space. It adjusts the y points until the distance differences between the two dimensions’ point distances are nearly zero.

Question 9

Statistics vs Machine Learning

Answer 9

Statistics makes assumptions about the nature of the data while machine learning addresses the data as it occurs in real world applications.

Question 10

ML for Data Science vs ML for Decision Making

Answer 10

ML for DS: generate presentable insights/hypotheses

ML for DM: focus on learning good autonomous decisions that can be taken for repetitive tasks without human intervention

Question 11

Data Science / ML lingo for row

Answer 11

Instance (can also say data point)

N means number of data points (instances)

Question 12

DS/ML lingo for column

Answer 12

Feature (can also say attribute)

D means number of features (attributes)

Question 13

Metadata

Answer 13

Description of the data in the dataset. What it is actually about.
Eg: description of dataset, features and instances

“Data about the data”

Question 14

Network Dataset Adjacency Matrix

Answer 14

Tabular structure of size NxN can be stored to represent Network Dataset. Checks for connection between each data point.

Question 15

Large Datasets

Answer 15

Datasets whose size is too large to be processed with classical techniques (eg single computers, standard software). Must synchronize the model between different machines (computers)

Question 16

Two level data analysis

Answer 16

First layer of data analysis may be performed on non-sensitive public data for the learning of the model. The data representations learned will then be applied to the second layer, private datasets for analysis to extract useful insights.