Machine Learning revision

Question 1

What are the benefits of preprocessing data / data wrangling?

Answer 1

• Transform raw data into a state which the machine can understand and interpret easily.
• Remove redundant information ~noise.
• Spot outliers and deal with them to make sure the training is effective and not skewed.
• Data in real world is not perfect.

Question 2

What is a feature?

Answer 2

A feature is an individual measurable property or characteristic of a phenomenon being observed. Features can be categorical or numerical.

Question 3

Name some common steps for data preprocessing in DS/ML?

Answer 3

• Taking care of the missing data
• Encoding Categorical Data
• Feature Scaling

Question 4

Which library to use for taking care of missing data? Recall the entire code.

Answer 4

from sklearn.impute import SimpleImputer

imputer = SimpleImputer(

missing_values = np.nan,

strategy = ‘mean’)

imputer.fit(X[:, a:b])

X[:, a:b] = imputer.transform(X[X:, 1:3])

Question 5

What are some strategies to take care of missing data apart from averaging?

Answer 5

1. Mean, median, Mode
2. Deleting missing data (delete full row, deleting the variable (not recommended)).
3. Time Series Specific Methods (Last Observation Carried Forward (LOCF) & Next Observation Carried Backward (NOCB), Linear Interpolation (not good for data with seasonal oscillations), Seasonal Adjustment + Linear Interpolation).
4. Use Regression
5. Multiple Imputation (question to explain what it is.)
6. Use k-NN Classification to impute categorial variables.
7. Imputation using Deep Learning - Datawig

Question 6

What is multiple imputation method for taking care of missing data?

Answer 6

Multiple ImputationImputation:

• Impute the missing entries of the incomplete data sets m times (m=3 in the figure). Note that imputed values are drawn from a distribution. Simulating random draws doesn’t include uncertainty in model parameters. Better approach is to use Markov Chain Monte Carlo (MCMC) simulation. This step results in m complete data sets.
• Analysis: Analyze each of the m completed data sets.
• Pooling: Integrate the m analysis results into a final result

Question 7

Types of missing data?

Answer 7

3- MCAR, MAR, NMAR :-

Missing completely at random (MCAR): a certain missing value has nothing to do with its hypothetical value and with the values of other variables.

Missing at random (MAR): Missing at random means that the propensity for a data point to be missing is not related to the missing data, but it is related to some of the observed data

Not missing at random (NMAR): missing value is dependent on some other variable’s value ex. People with high salaries don’t always want to reveal their salaries.

Difference between MCAR and MAR:

For example, if high school GPA data is missing randomly across all schools in a district, that data will be considered MCAR. However, if data is randomly missing for students in specific schools of the district, then the data is MAR.

Question 8

How to encode categorical data? Which library to use? Provide code:-

Answer 8

OneHotEncoding- to transform n categories to n-dimensional vector.

Encoding independent variables:-

import sklearn.compose import ColumnTransformer

from sklearn.preprocessing import OneHotEncoder

ct = ColumnTransformer(

transformers = [(‘encoder’,

                              OneHotEncoder(), 

                              [specific the columns])], 

remainder = ‘passthrough’)

X = np.array(ct.fit_transform(X))

Encoding yes-no variable:-

from sklearn.preprocessing import LabelEncoder

le = LabelEncoder()

y = le.fit_transform(y)

Question 9

What are the two types of feature scaling? Give their formulas. Code one of them.

Answer 9

Standardisation and normalisation
Standardisation: X = (X - mean(X))/s.d.(X)
Normalisation: X = (X- min(X))/(max(X) - min(X))

Standardisation:
from sklearn.preprocessing import StandardScaler
sc = StandardScalar()
X = sc.fot_transform(X)

Question 10

Benefits of Splitting the dataset to train and test set

Answer 10

To make sure that the hasn’t been overfitting.

Question 11

Code to split the dataset to train and test set

Answer 11

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_testsplit(X, y, test_size = between 0 and 1, random_state = optional)

Question 12

What are some types of Regression?

Answer 12

1. Simple Linear
2. Multiple Linear
3. Polynomial Linear
4. Support Vector
5. Decision Tree
6. Random Forest

Question 13

Train Linear regression

Answer 13

from sklearn.linear_model import LinearRegression
regressor = LinearRegression()
regressor.fit(X_train, y_train)

NOTE: Almost same as Linear Regression but not this uses LinearRegression and not Linear Regressor.

Question 14

What are the five methods of building models?

Answer 14

1. All in
2. Backward Elimination
3. Forward Selection
4. Bidirectional Elimination
5. Score Comparison

Question 15

Steps for Backward Elimination

Answer 15

1. Select a significance level to stay in the model (Default: 5%)
2. Fit the full model with all possible predictors.
3. Consider the predictor with the highest P-value. If, p>sig. level, go to step 4, otherwise finish.
4. Remove the predictor.
5. Fit the model without this variable. GO back to step 3.

Question 16

Train Multiple linear Regression

Answer 16

from sklearn.linear_model import LinearRegressor
regressor = LinearRegressor()
regressor.fit(X_train, y_train)

NOTE: Almost same as Linear Regression but not this uses LinearRegressor and not Linear Regression.

Question 17

Train Polynomial Linear Regression

Answer 17

Idea: treat polynomial as multiple linear regression with x2 = x^2, x3 = x^3 ….. So, we can create a matrix for powered features.

from sklearn.preprocessing import LinearRegression, PolynomialFeatures

poly_reg = PolynomialFeatures(degree = 2)
X_poly = poly_reg.fit_transform(X)
lin_reg_2 = LinearRegression()
lin_reg_2.fit(X_poly, y)

Question 18

reshape a 1D array to a 2D array

Answer 18

Let y be a 1D array.

y = y.reshape(len(y), 1)

Question 19

Train the Support Vector Regression Model

Answer 19

Note: It is needed to use Standard Scaling in SVR.

from sklearn.svm import SVR
regressor = SVR(kernel = ‘rbf’) # select your own ketnel
regressor.fit(X, y)

Question 20

train the Decision Tree Regression model

Answer 20

from sklearn.tree import DecisionTreeRegressor
regressor = DecisionTreeRegressor()
regressor.fit(X, y)

Question 21

Which regression model(s) requires the data to be standardised?

Answer 21

SVR

Question 22

train the Random Forest Regression model

Answer 22

from sklearn.ensemble import RandomForestRegressor
regressor = RandomForestRegressor(n_estimators = 10)
regressor.fit(X, y)

Question 23

Evaluating the Regression models

Answer 23

R squared (R^2)
A better one is -> Adjested (R^2)
code:-
from sklearn.metrics import r2_score
r2_Score(y_test, y_pred)

Question 24

Train Logistic Regression

Answer 24

data should be scaled.

from sklearn.linear_model import LogisticRegression
classifier = LogisticRegression()
classifier.fit(X_train, y_train)

Question 25

Why do we standardise data?

Answer 25

We standardise data to ensure that one variable in data doesn’t have a biased influence on the result from the start (just because its value is larger than all the other values).

Question 26

What can we use to evaluate the results of a classification model? Write code as well.

Answer 26

Confusion Matrix and accuracy score, CAP curve

from sklearn.metrics import comfusion_matrix, accuracy_score
cm = confusion_matrix(y_test, y_pred)
print(cm)
accuracy_score(y_test, y_pred)

Question 27

Code to visualize Classification models

Answer 27

from matplotlib.colors import ListedColormap
X_set, y_set = fc.inverse_transform(X_train, y_train)
X1, X2 = np.meshgrid(
np.arrange(start=X_set[:, 0].min() - 10,
stop=X_set[:,0].max() + 10, step = 0.25),
np.arrange(start=X_set[:, 1].min() - 10,
stop=X_set[:,1].max() + 10, step = 0.25))
plt.ccontourf(X1, X2,
classifier.predict(fc.transform(np.array([X1.ravel(),
X2.ravel()]).T))>reshape(X1.shape),
alpha = 0.75,
cmap = ListedColormap([‘red’, ‘green’]))
plt.xlim(X1.min(), X1.max())
plt.ylim(X2.min(), X2.max())
for i,j in enumerate(np.unique(y_set)):
plt.scatter(X_set[y_set == j,0], X_set[y_set == j, 1],
c = ListedColormap((‘red’, ‘green’))[i], label = j)
plt.title(‘…’)
plt.xlabel(‘…’)
plt.ylabel(‘…’)
plt.legend()
plt.show()

Question 28

Train the K_NN model Classification model

Answer 28

Data should be standardised

from sklearn.neighbors import KNeighborsClassifier
classifier = KNeighborsClassifier(n_neighbors = 5,
metric = ‘minkowski,
p = 2)
classifier.fit(X_train, y_train)

Question 29

Train the Support Vector Machine (SVM) Classification model

Answer 29

Data should be standardised

from sklearn.svm import SVC
classifier = SVC(kernel = ‘linear’) #’poly’/’rbf’/’sigmoid’ etc.
classifier.fit(X_train, y_train)

Question 30

code for the Naive-Bayes Classification model

Answer 30

Data should be standardised

from sklearn.naive_bayes import GaussianNB
classifier = GaussianNB()
classifier.fit(X_train, y_train)

Question 31

Code for the Decision Tree Classification model

Answer 31

Data should be standardised

from sklearn.tree import DecisionTreeClassifier
classifier = DecisionTreeClassifier(criterion = ‘entropy’)
classifier.fit(X_train, y_train)

Question 32

Code for the Random Forest Classification model

Answer 32

Data should be standardised

from sklearn.ensemble import RandomForestClassifier
classifier = RandomForestClassifier(
      n_estimators = 10,
     criterion = 'entropy')
classifier.fit(X_train, y_train)

Question 33

List some types of Classification Models

Answer 33

• Logistic Regression model
• K-NN
• Support Vector Machine Classification
• Naive-Bayes
• Decision Tree
• Random Forest

Question 34

Types of Clustering

Answer 34

• K-Means Clustering (K-Means++ avoids random initialisation trap).
• Hierarchal Clustering
  a. Agglomerative (bottom-up approach)
  b. Divisive (top-to-bottom approach)

Question 35

K-Means++ Clustering Logic

Answer 35

1. Choose the number K of clusters
2. Select at random K points, the centroids (not necessarily from your dataset).
3. Assign each data point to the closest centroid -> forming K clusters.
4. Compute and place the new centroid of each cluster.
5. Reassign each data point to the new closest centroid. If reassignment took place, go to step 4, otherwise FINSIH.

Question 36

Clustering is supervised or unsupervised model?

Answer 36

Clustering is an unsupervised model!

Question 37

Train K-Means++ Model

Answer 37

from sklearn.cluster import KMeans
kmeans = KMeans(n_clusters = 5, init = ‘k-means++’)
y_kmeans = kmeans.fit_predict(X)

Question 38

Identify the right number of clusters using WCSS (elbow method)

Answer 38

from sklearn.cluster import KMeans
wcss = []
for i in range(1, 11):  #11 not fixed, depends on the data.
     kmeans = KMeans(n_clusters = i, init = 'k-means++')
     kmeans.fit(X)
     wcss.append(kmeans.inertia_)
plt.plot(range(1, 11), wcss)
plt.title(...)
plt.xlabel(...)
plt.ylabel(...)
plt.show()

Question 39

Logic for Agglomerative (bottom-up) Clustering

Answer 39

1. Make each data point a single-point cluster, forming N clusters
2. Take two closest data points and make them one cluster => N-1 clusters
3. Take two closest cluster and make them one cluster => N-2 cluster
4. Repeat Step 3till only one cluster in left. FINISH.
5. Use dendrograms to determine the number of clusters

Question 40

train Hierarchal Agglomerative Clustering

Answer 40

‘ward’ is the best to minimise variance

from sklearn.cluster import AgglomerativeClustering
hc = AgglomerativeClustering(n_clusters = 5,
affinity = ‘euclidean’,
linkage = ‘ward’)
y_hc = hc.fit.predict(X)

Question 41

Code for Dendrograms

Answer 41

import scipy.cluster.hierarchy as sch
dendrogram = sch.dendrogram(sch.linkage(X,method='ward'))
plt.title(...)
plt.xlabel(...)
plt.ylabel(...)
plt.show()

Question 42

Ways to evaluate clustering models

Answer 42

sklearn Clustering performance evaluation :-

1. Adjusted Rand Index (ARI requires knowledge of the ground truth classes)
2. Silhoutte Coefficient (when ground truth is unknown)

there are a lot of ways to do both cases, and the sklearn library should be considered before choosing which one to use.

Question 43

Types of Association Rule Learning

Answer 43

People who did A, also did B.

1. APRIORI
2. ECLAT

Question 44

APRIORI logic

Answer 44

1. Set a minimum support and confidence
2. Take all the subsets in transactions having higher support than minimum support.
3. Take all the rules of these subsets having higher confidence than minimum confidence
4. Sort the rules by decreasing lift.

#note
Support(event A) = (Times event A occurs/ Times all events occur)

Confidence(event A -> event B) =
(Times event B followed by event A occurs/ times event A happened)

Lift(A -> B) = Confidence(A -> B) / Support(B)

Question 45

write the formulas for support, confidence, and lift in association learning.

Answer 45

Support(event A) = (Times event A occurs/ Times all events occur)

Confidence(event A -> event B) =
(Times event B followed by event A occurs/ times event A happened)

Lift(A -> B) = Confidence(A -> B) / Support(B)

Question 46

train APRIORI

Answer 46

transactions = []
for i in range(0, len(dataset[0])): #columns
transactions.append([str(dataset.values[i,j]) for j in range (0, len(dataset)) #rows

from apyori import apriori
rules = apriori(transactions, min_support = 0.003, min_confidence = 0.2, min_lift = 3, min_length = 2)

Question 47

ECLAT

Answer 47

ECLAT uses support only.

Can be implemented using apyrori only by only including min_support as an argument.

Question 48

Ways to implement Reinforcement Learning

Answer 48

1. Upper Confidence Bound
2. Thompson Sampling

Code implemented on own- see downloaded documents.

Question 49

Natural Language Processing Steps

Answer 49

1. Clean the text
   i. get only the relevant words
   ii. get rid of words like ‘the’, ‘a’, ‘on’, ‘…’, etc.
   iii. can get rid of numbers unless they have significant
   impact.
   iv. Stemming. (transform words to their roots eg. Singing -> sing)
2. Bag of words model
3. Use Classification (common models: Naive-Bayes and Random Forest). // Standard Scaling won’t be needed as its mostly 0s and 1s.

Question 50

Types and subtypes of Dimensionality Reduction

Answer 50

1. Feature selection:-
   
   • backward elimination
   • forward selection
   • bidirectional elimination
   • score comparison etc.
2. Feature Extraction
   
   • Principal Component Analysis (PCA).
   • Linear Discriminant Analysis (LDA).
   • Kernel PCA.
   • Quadratic Discriminant Analysis (QDA).

Question 51

Feature Extraction: PCA

Answer 51

• PCA is an unsupervised model
• It is used for noise filtering, visualisation, feature extraction, stock market predictions, gene data analysis.
• PCA identifies patterns in data and detects the correlation between the variable.
• GOAL: Reduce the dimensions of a d-dimensional dataset by projecting it into a k-dimensional subspace (where k

Question 52

Feature Extraction: PCA Steps

Answer 52

1. Standardise the data
2. Obtain the Eigenvectors or Eigenvalues from the covariance matrix or correlation matrix, or perform Singular Vector Decomposition.
3. Sort eigenvalues in descending order and choose the k eigenvectors that correspond to the k largest eigenvalues where k is the number of dimensions of the new feature subspace (k <= d)
4. Construct the projection matrix W from the selected k eigenvectors.
5. Transform the original dataset X via W to obtain a k-dimensional feature subspace Y.

Question 53

Code for PCA

Answer 53

Apply PCA generally after feature scaling and before training the model

Code:-
from sklearn.decomposition import PCA
pca = PCA(n_components = 2)
X_train = pca.fit_transform(X_train)
X_test = pca.transform(X_test)

…train the model…

Question 54

Feature Extraction: LDA

Answer 54

• commonly used
• supervised
• Goal of LDA is to project a feature space onto a small subspace k while maintaining the class-discriminatory information.

Question 55

PCA vs LDA

Answer 55

• Both PCA and LDA are linear transformation techniques used for dimensional reduction.
• LDA => supervised because of the relation to the
  dependent variable.
  PCA=> unsupervised.
  -

Question 56

Feature Extraction: LDA Steps

Answer 56

1. Compute the d-dimensional mean vectors for the different classes from the dataset.
2. Compute the scatter matrices (in-between-class and within_class scatter matrix).
3. Compute the eigenvectors (e1, e2, …, eN) and considering eigenvalues (l1, l2, …, lN) for the scatter matrices.
4. Sort the eigenvectors by decreasing eigenvalues and choose k eigenvectors with the largest eigenvalues to forma. dxk-dimensional matrix W(where every column represents an eigenvector).
5. Use this dxK eigenvectors matrix to transform the samples onto the new subspace => Y = XxW (where X is a non-dimensional matrix representing the n samples, and y are the transformed nxk dimensional samples in the subspace.

Question 57

Code for LDA

Answer 57

after feature scaling

froms sklearn.discriminant_analysis import LinearDiscriminantAnalysis as LDA
lda = LDA(n_components = 2)
X_train = lda.fit_transform(X_train, y_train)
X_test = lda.transform(X,test)

..train the model ….

Question 58

Code for Kernel PCA

Answer 58

Apply PCA generally after feature scaling and before training the model

Code:-
from sklearn.decomposition import KernelPCA
pca = KernelPCA(n_components = 2, kernel = ‘rbf)
X_train = pca.fit_transform(X_train)
X_test = pca.transform(X_test)

…train the model…

Question 59

k-Fold Cross Validation

Answer 59

• to avoid the case that we just got lucky.
• divides training_set to smaller parts and tests the accuracies.

-After training the model

Code:
from sklearn.model_selection import cross_val_score
accuracies = cross_val_score(
estimator = NAME_OF_MODEL,
X = X_train,
y = y_train,
cv = 10) #10 is most common.
print(“Accuracy:{%.2f}%”.format(accuracies.mean()100))
print(“Std Dev:{%.2f}%”.format(accuracies.std()100))

Question 60

Grid Search (for hyper-parameters)

Answer 60

After training the model and applying k-Fold Cross Validation:-

For Kernel SVM:-
from sklearn.model_selection import GridSearchCV
parameters = #what we want to tune [
   {'c': [0.25, 0.5, 0.75, 1],
    'kernel': ['linear']},
   {'c': [0.25, 0.5, 0.75, 1],
    'kernel':['rbf'],
    'gamma':[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]}]

grid_search = GridSearchCV(estimator = classifier,
param_guide = parameters,
scoring = ‘accuracy’,
cv = 10,
n_jobs = -1)

grid_search.fit(X_train, y_train)
best_accuracy = grid_search.best_score_
best_parameters = grid_search.best_params_
print(“Best Accuracy:{.2f}%”.format(best_accuracy * 100))
print(“Best Parameters:”, best_parameters