CHAP 5 : ML Intro and KNN

Question 1

What is the difference between traditional programming and machine learning?

Answer 1

In traditional programming, computers are merely following instructions from the program. However, in machine learning, computers learn from experiences just as a human does and makes decisions.

Traditional programming:
- Input : Data, rules (logic)
- Output : Answers

Machine learning:
- Input : Data, answers
- Output : Logic

Question 2

What are the 3 categories that machine learning can be broadly classfied into?

Answer 2

1. Supervised learning
2. Unsupervised learning
3. Reinforcement learning

Question 3

What kinds of tasks are solved in supervised learning? Give examples.

Answer 3

1. Classsification – Object classification in surveillance videos / photos etc
2. Regression – Prediction of house prices

Question 4

What is the goal of supervised learning and what kind of data is passed into supervised learning?

Answer 4

Goal of supervised learning algorithm is to learn patterns in the data and build a general set of rules to map input to the class / event.

Human-labelled data is passed as input

Question 5

What are the 3 stages of supervised learning model?

Answer 5

1. Training
2. Testing / Validation
3. Classification / prediction

Question 6

What kinds of tasks are solved in unsupervised learning? Give examples.

Answer 6

1. Clustering – grouping of similar customer profiles (aka customer segmentation for marketing purposes)
2. Dimensionality reduction – finding key features in data
3. Anormaly detection – detecting fraud credit card transaction

Question 7

What is dimensionality reduction? [not so impt]

Answer 7

transformation of data from a high-dimensional space into a low-dimensional space so that the low-dimensional representation retains some meaningful properties of the original data

Question 8

What is the idea behind reinforcement learning?

Answer 8

• These algorithm maps situations to actions that yield maximum rewards
• For every action, there is a reward defined by the user
• The algorithm learns to find the action that maximises the reward

Question 9

Give examples of reinforcement learning.

Answer 9

AI for games (e.g. ATARI Game AIs / game bots)
Self-driving car

Question 10

What are the 3 major tasks in ML?

Answer 10

1. Classification
2. Regression
3. Clustering

Question 11

What kinds of data are used in ML algorithms ? [2]

Answer 11

1. Numerical
2. Categorical

Question 12

What are the 2 kinds of numerical data? Give examples.

Answer 12

1. Discrete data – Shows the count that involves only integers, cannot subdivide the values into parts (e.g. no of children)
2. Continuous data – can be meaningully divided into finer levels, measured on a scale / continuum and have almost any numeric value (e.g. weight and height)

Question 13

What are the 2 kinds of categorial data? Give examples.

Answer 13

1. Nominal data – used for labelling variables, has no order (sometimes referred to as “labels”) – e.g. Gender
2. Ordinal data – values of ordinal data have some natural ordering – e.g. rating product on a scale of 1-5, clothes sizing (small, medium, large)

Question 14

Higher dimensional data provides more details but incurs more computation in Machine Learning. True or False?

Answer 14

True

Question 15

What are the characterisitics of kNN algorithm?

Answer 15

kNN is a
1. Supervised – need labeled training data
2. Non-parametric – no assumption made on data distribution

3. Lazy learning – no need for training the data, no model generated

algorithm

Question 16

What is the key idea behind kNN?

Answer 16

It is used to find the k-most SIMILAR datapoints from the dataset.

Question 17

How does kNN algo work? (How is the class of new data point determined?)

Answer 17

• Algo works based on the majority vote of the k nearest neighbours class
• New datapoint is assigned to the most common class of the k-nearest neighbours

Question 18

What are similarity measures in kNN?

Answer 18

A similarity measure is a distance measure with dimensions representing features of the objects.

Question 19

What are the kinds of similarity measures (distance metrics) used in kNN?

Answer 19

1. Euclidean distance **
2. Manhattan distance **
3. Minkowski distance
4. Hamming distance (same as manhattan distance but binary values)

Question 20

How is similarity measured?

Answer 20

Similarity is measured in the range 0 to 1.

For 2 features X and Y,
– if similarity score of X and Y = 1, X == Y
– if similarity score of X and Y = 0 , X != Y

Question 21

Why is feature scaling necessary?

Answer 21

Since ML algorithms takes only the magnitude of features neglecting the units, there is a chance where higher weightage is given to features with higher magnitude.

With feature scaling, features will have equal magnitude.

Question 22

What are the 2 ways we can scale features? When are they used?

Answer 22

1. Standardisation : x’ = (x-mean) / (stdev)

– Used when data has varying range and when data follows a distribution

2. Nomalisation : x’ = [x-min(x)] / [max(x)-min(x)]

– Used when there is no assumption made on the distribution of data

Question 23

kNN works well with large and higher dimension data. Is the statement true or false?

Answer 23

False

Question 24

Which type of data do the Cumulative Average Points (CAP) fall under?

Answer 24

Continuous

Question 25

Which type of data do school rankings fall under (NUS, NTU, SMU rankings)?

Answer 25

Ordinal

Question 26

Given the same dataset and input, the evaluation of multiple runs of kNN algorithm will yield different results. Is the statement true or false?

Answer 26

False

Question 27

The manhattan distance is the straight line distance between two points. Is the statement true or false?

Answer 27

False. That is euclidean distance

Question 28

Euclidean distance is a variation of Minkowski distance where the order of the norm is 2. Is the statement true or false?

Answer 28

True

Question 29

Hamming Distance can be used on (x,y) co-ordinate values. Is the statement true or false?

Answer 29

False, used on binary values

Question 30

Feature scaling methods should be used in KNN as the distance metrics are used in the algorithm.

Answer 30

True