AI

Question 1

What are the 2 types of data ?

Answer 1

Numerical Data and Categorical Data.

Question 2

What kind of value does Numerical or Continuous data accept ?

Answer 2

Can accept any value within a finite or infinite interval (e.g., height, weight, temperature, blood glucose, …).

Question 3

What are the 2 types of Numerical or Continuous data ?

Answer 3

Interval and ratio.

Question 4

Describe data on an interval scale.

Answer 4

Can be added and subtracted but cannot be meaningfully multiplied or divided because there is no true zero. For example, we cannot say that one day is twice as hot as another day.

Question 5

Describe data on a ratio scale.

Answer 5

Has true zero and can be added, subtracted, multiplied or divided (e.g., weight).

Question 6

Categorical or Discrete variable is the one that has ….. .

Answer 6

two or more categories (values).

Question 7

What are the 2 types of categorical variables ?

Answer 7

Nominal and ordinal.

Question 8

Describe Nominal variables.

Answer 8

Has no intrinsic ordering to its categories. For example, gender is a categorical variable having two categories (male and female) with no intrinsic ordering to the categories.

Question 9

Describe Ordinal variables.

Answer 9

Has a clear ordering. For example, temperature as a variable with three orderly categories (low, medium and high).

Question 10

What is a frequency table ?

Answer 10

Is a way of counting how often each category of the variable in question occurs. It may be enhanced by the addition of percentages that fall into each category.

Question 11

What is Encoding or continuization ?

Answer 11

Is the transformation of categorical variables to binary or numerical counterparts. An example is to treat male or female for gender as 1 or 0. Categorical variables must be encoded in many modeling methods (e.g., linear regression, SVM, neural networks).

Question 12

What are the 2 types of encoding ?

Answer 12

Binary and Target-based.

Question 13

What is Binning or discretization ?

Answer 13

Is the process of transforming numerical variables into categorical counterparts.
An example is to bin values for Age into categories such as 20-39, 40-59, and 60-79.

Question 14

Numerical variables are usually discretized in the modeling methods based on ….. .

Answer 14

frequency tables (e.g., decision trees).

Question 15

Binning may improve accuracy of the predictive models by ….. or ….. .

Answer 15

reducing the noise, non-linearity.

Question 16

What is a Dataset ?

Answer 16

Is a collection of data, usually presented in a tabular form. Each column represents a particular variable, and each row corresponds to a given member of the data.

Question 17

Alternatives for columns: ….., ….., ….. .

Answer 17

Fields, Attributes, Variables.

Question 18

Alternatives for rows: ….., ….., ….., ….., ….., ….. .

Answer 18

Records, Objects, Cases, Instances, Examples, Vectors.

Question 19

Alternatives for values: ….. .

Answer 19

Data.

Question 20

In predictive modeling, ….. or ….. are the input variables

Answer 20

predictors, attributes.

Question 21

In predictive modeling, ….. or ….. is the output variable

Answer 21

target, class attribute.

Question 22

In predictive modeling, the output variable value is determined by ….. and ….. .

Answer 22

the values of the predictors, function of the predictive model.

Question 23

Pattern recognition predicts the future by ….. .

Answer 23

means of modeling.

Question 24

What is Predictive modeling ?

Answer 24

Is the process by which a model is created to predict an outcome.

Question 25

If the outcome is categorical it is called ..... .

Answer 25

Classification.

Question 26

If the outcome is numerical it is called ..... .

Answer 26

Regression.

Question 27

What is Descriptive modeling or clustering ?

Answer 27

Is the assignment of observations into clusters so that observations in the same cluster are similar.

Question 28

What is Classification ?

Answer 28

Is a predicting the value of a categorical variable (target or class) by building a model based on one or more numerical and/or categorical variables (predictors or attributes).

Question 29

What is ZeroR classifier ?

Answer 29

Is the simplest classification method which relies on the target and ignores all predictors.

Question 30

ZeroR classifier simply predicts the ..... .

Answer 30

Majority category (class).

Question 31

Although there is no predictability power in ZeroR, it is useful for ..... .

Answer 31

determining a baseline performance as a benchmark for other classification methods.

Question 32

What is the ZeroR classifier Algorithm ?

Answer 32

Construct a frequency table for the target and select its most frequent value.

Question 33

What is OneR classifier ?

Answer 33

Short for "One Rule", is a simple classification algorithm that generates one rule for each predictor in the data, then selects the rule with the smallest total error as its "one rule".

Question 34

To create a rule for a predictor, we ..... .

Answer 34

Construct a frequency table for each predictor against the target.

Question 35

OneR produces rules only slightly less accurate than state-of-the-art classification algorithms while ..... .

Answer 35

producing rules that are simple for humans to interpret.

Question 36

What is the OneR Classifier Algorithm ?

Answer 36

For each predictor, For each value of that predictor, make a rule as follows; Count how often each value of target (class) appears. Find the most frequent class. Make the rule assign that class to this value of the predictor. Calculate the total error of the rules of each predictor. Choose the predictor with the smallest total error.

Question 37

A low total error means ..... .

Answer 37

a higher contribution to the predictability of the model.

Question 38

A is a random variable if:

Answer 38

- A denotes something about which we are uncertain. | - perhaps the outcome of a randomized experiment.

Question 39

What is P(A) ?

Answer 39

The fraction of possible worlds in which A is true.

Question 40

What is the set of possible worlds called ?

Answer 40

Sample space (S).

Question 41

The Naive Bayesian (NB) classifier is based on ..... .

Answer 41

Bayes’ theorem with independence assumptions between predictors.

Question 42

A Naive Bayesian model is easy to build, with no complicated iterative parameter estimation which makes it particularly useful for ..... .

Answer 42

very large datasets.

Question 43

Why does the Naive Bayesian classifier often do surprisingly well and is widely used ?

Answer 43

Because it often outperforms more sophisticated classification methods.

Question 44

What is the Naive Bayes classifier algorithm ?

Answer 44

``` Bayes theorem provides a way of calculating the posterior probability, P(c|x), from P(c), P(x), and P(x|c). Naive Bayes classifier assume that the effect of the value of a predictor (x) on a given class (c) is independent of the values of other predictors. This assumption is called class conditional independence. ```

Question 45

How to solve the The zero-frequency problem ?

Answer 45

Add 1 to the count for every attribute value-class combination (Laplace estimator) when an attribute value (Outlook=Overcast) doesn’t occur with every class value (Play Golf=no).

Question 46

What to do when working with numerical predictors ?

Answer 46

- Numerical variables need to be transformed to their categorical counterparts (binning) before constructing their frequency tables. - The other option we have is using the distribution of the numerical variable to have a good guess of the frequency. - For example, one common practice is to assume normal distributions for numerical variables.

Question 47

What are the 2 parameters that defines The probability density function (PDF) for the normal distribution ?

Answer 47

Mean and standard deviation.

Question 48

What is Decision Tree Classification ?

Answer 48

- Decision tree builds classification or regression models in the form of a tree structure. - It breaks down a dataset into smaller and smaller subsets while at the same time an associated decision tree is incrementally developed. - The final result is a tree with decision nodes and leaf nodes. - Decision trees can handle both categorical and numerical data.

Question 49

Describe a decision node.

Answer 49

A decision node (e.g., Outlook) has two or more branches (e.g., Sunny, Overcast and Rainy).

Question 50

Describe a leaf node.

Answer 50

Leaf node (e.g., Play) represents a classification or decision.

Question 51

The topmost decision node in a tree which corresponds to the best predictor is called ..... .

Answer 51

root node.

Question 52

What is the decision tree classifier algorithm ?

Answer 52

The core algorithm is called ID3 employs a top-down, greedy search through the space of possible branches with no backtracking. ID3 uses Entropy and Information Gain to construct a decision tree.

Question 53

What is Entropy ?

Answer 53

- A decision tree is built top-down from a root node and involves partitioning the data into subsets that contain instances with similar values (homogenous). - ID3 algorithm uses entropy to calculate the homogeneity of a sample. If the sample is completely homogeneous the entropy is zero and if the sample is an equally divided it has entropy of one.

Question 54

The information gain is based on ..... .

Answer 54

the decrease in entropy after a dataset is split on an attribute.

Question 55

Constructing a decision tree is all about ..... .

Answer 55

Finding attribute that returns the highest information gain.

Question 56

How to transform decision tree to decision rules ?

Answer 56

By mapping from the root node to the leaf nodes one by one.

Question 57

What are some decision tree issues ?

Answer 57

- Working with continuous attributes (binning). - Avoiding overfitting - Super Attributes (attributes with many values). - Working with missing values.

Question 58

What is Logistic Regression ?

Answer 58

Logistic regression predicts the probability of an outcome that can only have two values (i.e. a dichotomy).

Question 59

In logistic regression, what is The prediction based on ..... .

Answer 59

The use of one or several predictors (numerical and categorical).

Question 60

What are the 2 reasons why a linear regression is not appropriate for predicting the value of a binary variable ?

Answer 60

- A linear regression will predict values outside the acceptable range (e.g. predicting probabilities outside the range 0 to 1). - Since the dichotomous experiments can only have one of two possible values for each experiment, the residuals will not be normally distributed about the predicted line.

Question 61

A logistic regression produces a logistic curve, which is limited to values between ..... .

Answer 61

0 and 1.

Question 62

Logistic regression is similar to a linear regression, but the curve is constructed using ..... , rather than the probability.

Answer 62

The natural logarithm of the “odds” of the target variable.

Question 63

..... is the method used to estimate coefficients for the best fit line in linear regression.

Answer 63

Ordinary least square regression.

Question 64

logistic regression uses ..... to obtain the model coefficients that relate predictors to the target.

Answer 64

Maximum likelihood estimation (MLE).

Question 65

A ..... value is available to indicate the adequacy of the regression model.

Answer 65

pseudo R2.

Question 66

What is a Likelihood ratio test ?

Answer 66

Is a test of the significance of the difference between the likelihood ratio for the baseline model minus the likelihood ratio for a reduced model.

Question 67

A ..... difference between the likelihood ratio for the baseline model minus the likelihood ratio for a reduced model.

Answer 67

model chi-square

Question 68

..... is used to test the statistical significance of each coefficient (b) in the model (i.e., predictors contribution).

Answer 68

Wald test

Question 69

What is the Core of ML ?

Answer 69

Making predictions or decisions from Data.

Question 70

What are the 5 principals of Representation ?

Answer 70

- Coverage. - Concision. - Directness. - Templates. - Histograms.

Question 71

What is K nearest neighbors (KNN)?

Answer 71

Is a simple algorithm that stores all available cases and classifies new cases based on a similarity measure (e.g., distance functions).

Question 72

..... has been used in statistical estimation and pattern recognition already in the beginning of 1970’s as a non-parametric technique. 

Answer 72

K nearest neighbors (KNN).

Question 73

What is the KNN algorithm ?

Answer 73

- A case is classified by a majority vote of its neighbors, with the case being assigned to the class most common amongst its K nearest neighbors measured by a distance function. - If K = 1, then the case is simply assigned to the class of its nearest neighbor.

Question 74

What are the 3 distance functions ?

Answer 74

- Euclidean - Manhattan - Minkowski

Question 75

All three distance measures are only valid for ..... variables.

Answer 75

Continuous.

Question 76

In the instance of categorical variables the ..... distance must be used. 

Answer 76

Hamming.

Question 77

Choosing the optimal value for K is best done by first ..... .

Answer 77

Inspecting the data.

Question 78

In general, a ..... K value is more precise as it reduces the overall noise but there is no guarantee.

Answer 78

Large.

Question 79

..... is a way to retrospectively determine a good K value by using an independent dataset to validate the K value.

Answer 79

Cross-validation.

Question 80

Historically, the optimal K for most datasets has been between ..... and ..... . That produces much better results than 1NN.

Answer 80

3, 10.

Question 81

One major drawback in calculating distance measures directly from the training set is in the case where ..... or ..... .

Answer 81

Variables have different measurement scales, there is a mixture of numerical and categorical variables.

Question 82

What is Linear Discriminant Analysis (LDA) ?

Answer 82

LDA is a classification method based upon the concept of searching for a linear combination of the variables that best separates two classes, It is simple, mathematically robust and often produces models whose accuracy is as good as more complex methods. 

Question 83

One way of assessing the effectiveness of the discrimination is to calculate ..... .

Answer 83

the Mahalanobis distance between two groups.

Question 84

In LDA model assessment what does a distance greater than 3 means ?

Answer 84

it means that in two averages differ by more than 3 standard deviations. It means that the overlap (probability of misclassification) is quite small.

Question 85

A simple linear correlation between the LDA model scores and predictors can be used to ..... .

Answer 85

test which predictors contribute significantly to the discriminant function.

Question 86

How to avoid overfitting ?

Answer 86

- Stop growing when data split not statistically significant. - Grow full tree, then post-prune.

Question 87

What are decision tree Pros ?

Answer 87

- Simple to understand and interpret. - Little data preparation and little computation. - Indicates which attributes are most important for classification.

Question 88

What are decision tree Cons?

Answer 88

- Learning an optimal decision tree is NP-complete. - Perform poorly with many classes and small data. - Computationally expensive to train. - Over-complex trees do not generalize well from the training data (overfitting).