Introduction to data analytics

Question 1

What are the 4 Vs of Big Data?

Answer 1

Volume
Velocity
Variety
Veracity

Question 2

What paradigm do big data scientists use?

Answer 2

Retrospective data mining with multiple hypotheses

Looking for patterns without a particular hunch

Question 3

Types of Data?

Answer 3

Structured Data

• SQL Database
• Excel Spreadsheet / CSV File

Unstructured Data

• Free Text Responses
• Doctor’s notes

Question 4

Typical Data Structures

Answer 4

Typical Datasets
- CSV 
- eXtensible Markup Language (XML)
- JavaScript Object Notation (JSON)
   Nested JSON in CSV
- SQL
- Excel Data Formats
Other
- .txt files for text
- RGB data for images

Question 5

What is Quantitative Data?

Answer 5

Numbers, such as -

• Temperature
• Heart rate
• Likert ratings

Question 6

What is Qualitative Data?

Answer 6

Text, such as -

• Surveys
• Interviews
• App comments / User Feedback

Question 7

What is Quantitative Analysis?

Answer 7

Qauntitative Statistical Analysis:

• Descriptive statistics: Mean and SD
• Inferential Statistics: t-tests

Question 8

What is Qualitative Analysis?

Answer 8

Thematic Analysis
Advanced methods:
- Text analytics: word embeddings
- Review mining

Question 9

Gartner Analytic Continuum

Answer 9

Descriptive Analytics
- Hindsight
Diagnostics Analytics 
- Insight
Predictive Analytics
- Foresight
Prescriptive Analytics

Increasing difficulty and value

Question 10

Typical Data Analytics Process

Answer 10

Data gathering/wrangling/linking -> data cleansing -> exploratory data analysis [EDA] -> supervised machine learning

EDA

• Data visualisation
• Association mining
• Unsupervised machine learning

Supervised Machine Learning

• Feature engineering
• Model building
• Model optimisation
• Model evaluation

Question 11

Machine Learning

Answer 11

Supervised ML

• Labelled data
• DL, SVM, Logit, Decision Trees, K-NN

Unsupervised ML

• Unlabeled Data
• Clustering, association rule mining

Semi-supervised ML
- Some labelled data

Question 12

The 5 Tribes of ML & the No free lunch theorem

Answer 12

Symbolists | Structure Inference | Production Rule System & Inverse Deduction
Connectionist | Estimating Parameters | Back propagation & Deep Learning
Bayesians | Weighing Evidence | HMM Graphical Model
Evolutionaries | Structure Learning | Genetic Algorithms & Evolutionary Programming
Analogisers | Mapping to Novelty | kNN and SVM

Question 13

The Neat and Scruffy Data Scientist

Answer 13

Neat: they care about the details and the ML methods
Scruffy: they care about the results and are somewhat ignorant of details and the methods