Lecture 7

Question 1

Random Variable

Answer 1

Refers to an element/event whose status is unknown

Question 2

Domain

Answer 2

The set of values a random variable can take

Question 3

Conditional probability

Answer 3

the chances that some outcome occurs given that another event has also occurred

Question 4

Joint Probability

Answer 4

The probability that a set of random variables will take a specific value

Question 5

What are three types of classifiers?

Answer 5

Instance based classifiers, generative, and discriminative

Question 6

Instance based classifiers

Answer 6

Use observations directly without models

e.g. k nearest neighbors

Question 7

Generative classifiers

Answer 7

build a generative statistical model

e.g. Bayes classifiers

Question 8

Discriminative classifiers

Answer 8

directly estimate a decision rule/boundary

e.g. decision trees

Question 9

Gaussian Naive Bayes classifier

Answer 9

assumes that features follow a normal distribution

Question 10

Multinomial Naive Bayes

Answer 10

each feature represents an integer count of something, like how often a word appears in a sentence

Question 11

Bernoulli Naive Bayes

Answer 11

Assumes your feature vectors are binary or continuous values which can be precisely split (binarized) with a predefined threshold

Question 12

Advantages of Naive Bayes classifiers

Answer 12

They are simple, work well with a small amount of training data, and the class with the highest probability is considered as the most likely class

Question 13

Disadvantage of Naive Bayes classifiers

Answer 13

Estimates parameters

Question 14

What is the complexity of a decision tree model determined by?

Answer 14

the depth of the tree

Question 15

What causes overfitting in decision trees?

Answer 15

Increasing the depth of the tree and thus increasing the number of decision boundries

Question 16

What is the aim of Bayesian Linear Regression?

Answer 16

Not to find the single “best’ value of the model parameters, but rather to determine the posterior distribution for the model parameters

Question 17

Advantages of Decision Trees

Answer 17

They are suitable for multiclass classification, the model is easily interpretable, it can handle numerical and categorical data, is non-linear, and can tolerate missing values

Question 18

Disadvantages of Decision Trees

Answer 18

They’re prone to overfitting without pruning, are weak learners, and singular trees do not make great predictions

Question 19

Which classification algorithms that we know are linear?

Answer 19

Logistic regression and

Linear SVMs

Question 20

Which classification algorithms that we know are nonlinear?

Answer 20

KNN
Neural Networks (Multi-Layer Perceptron)
Kernel SVMs
Naive Bayes
and Decision Trees

Question 21

What does the vertical line | in probability theory refer to?

Answer 21

given

e.g. p(A|B) is the probability of A given B.