B06 K-NN

Question 1

Some examples of Parametric models are:

Answer 1

• Linear Regression
• Logistic Regression
• Naive Bayes
• Simple Neural Networks

Question 2

Some examples of Non- Parametric models are:

Answer 2

• k - Nearest Neighbor
• Support Vector Machines
• Decision Trees

Question 3

A learning model that summarizes data with a set of
parameters of fixed size (independent of the number of
training examples) is called a __________. No
matter how much data you throw at a ___________, it won’t change its mind about how many
parameters it needs.”

Answer 3

parametric model
parametric
model

Question 4

Strengths of Parametric Models?

Answer 4

-Simpler: These methods are easier to
understand and the results are easy to
interpret.
-Speed: Parametric models are usually
very fast to train.
-Less Data: They do not require as much
training data and can work well even if
the fit to the data is not perfect.

Question 5

Weaknesses of Parametric Models?

Answer 5

-Constrained: By choosing a functional
form, these methods are highly
constrained to the specified form.
-Limited Complexity: The methods are
more suited to simpler problems.
-Poor Fit: In practice, the methods may
not always match the underlying
mapping function.

Question 6

A learning model that does not make strong
assumptions about the form of the mapping function is
called a ___________. By not making
assumptions, ____________ are free to learn
any functional form from the training data.

Answer 6

non-parametric model

Question 7

Weaknesses of Non-Parametic Models?

Answer 7

-More data: Require a lot more training
data to estimate the mapping
function.
-Slower: A lot slower to train, as they
often have far more parameters to
train.
-Overfitting: Have a higher risk of
overfitting against the training data

Question 8

Strengths of Non-Parametric Models?

Answer 8

-Flexibility: Capable of fitting a large
number of functional forms.
-Power: No assumptions (or weak
assumptions) about the underlying
function.
-Performance: Can result in higher
performance models for prediction.

Question 9

A class of non-parametric learning methods that do
not generate a model but instead make use of
verbatim training data for classification?

Answer 9

Lazy or instance-based learners or

rote learners

Question 10

The _________________algorithm gets its name
from the fact that it classifies an unlabeled observation
based on information about the _______labeled
________ of the observation.

Answer 10

k-Nearest Neighbor (k-NN)
k-nearest
neighbors

Question 11

Choosing the right K

A ____ reduces the impact of
noisy data but increases the risk of
ignoring important patterns

Answer 11

large K

Question 12

Choosing the right K

A _______ makes the model
susceptible to noise and/or outliers.

Answer 12

small K

Question 13

Note that the ______ the dataset, the ____
important the difference between two choices
for k becomes.

Answer 13

larger

less

Question 14

Strengths of K-NN?

Answer 14

-Simple and effective.
-Makes no assumptions about the
underlying data distribution.
-Training phase is very fast

Question 15

Weaknesses of K-NN?

Answer 15

-Does not produce a model.
-The selection of an appropriate
k is often arbitrary.
-Rather slow classification
phase.
-Does not handle missing, outlier
and nominal data well without
pre-processing.