Yet Another Deck

Question 1

Regression is a data mining task of predicting the value of the target (_) by building a model based on one of more predictors

Answer 1

numerical variable

Question 2

regression options

Answer 2

* decision tree (freq table) * multiple linear regression (co variance matrix) * k-nearest neighbor (similarity functions) * artificial neural networks (other) * support vector machine (other) Dinosaurs made Kites and Vikings. Natural theory of regression.

Question 3

The ID3 algorithm can be used to construct as decision tree for regression by

Answer 3

replacing information gain with standard deviation reduction

Question 4

the standard deviation reduction for ID3 regression is based on the

Answer 4

decrease in standard deviation after a dataset is split on an attribute

Question 5

constructing an ID3 decision tree is all about finding attribute that returns the

Answer 5

highest standard deviation reduction

Question 6

when building an ID3 regression decision tree, a branch with standard deviation of more than zero _

Answer 6

requires further splitting

Question 7

Decision trees. To stop splitting forever we need some termination criteria, for example, when the _ becomes smaller than a certain fraction of the _

Answer 7

standard deviation, standard deviation of for the full dataset (e.g. 5%)

Question 8

Decision trees. To stop splitting forever we need some termination criteria, for example, when too _

Answer 8

few instances remain in the branch (e.g. 3)

Question 9

Decision trees. To stop splitting forever we need some termination criteria. Then when the number of instances is more than one at a leaf node we _

Answer 9

calculate the average as the final value for the target

Question 10

Logistic regression predicts

Answer 10

the probability of an outcome than can only have two values (i.e. a dichotomy)

Question 11

The prediction for logistic regression is based on the use of one of several predictors (_ & _)

Answer 11

numerical, categorical

Question 12

A linear regression is not appropriate for predicting the value of a binary variable for two reasons (1) linear regression will

Answer 12

predict values outside the acceptable range (0 to 1)

Question 13

A linear regression is not appropriate for predicting the value of a binary variable for two reasons (2) since dichotomous experiments can only have one of two possible values for each experiment, the residuals will not

Answer 13

be normally distributed about the predicted line

Question 14

Logistic regression produces a _ which is _

Answer 14

logistic curve, limited to the values between 0 and 1

Question 15

logistic regression is similar to linear regression but the curve is constructed using the natural logarithm of the _ of the target variable, rather than the probability

Answer 15

odds

Question 16

logistic regression is similar to linear regression but the predictors do not

Answer 16

have to be normally distributed or have equal variance in the group

Question 17

Just as ordinary least square regression is the method used to estimate coefficients for the best fit line in linear regression, logistic regression uses _ to obtain the model coefficients that relate predictors to the targer

Answer 17

maximum likelihood estimation (MLE)

Question 18

Association rules is a pattern that states when an event occurs _

Answer 18

another event occurs with a certain probability

Question 19

Most instance-based learners use:

Answer 19

Euclidean distance

Question 20

Alternative to Euclidean distance

Answer 20

Manhattan, City-Block

Question 21

It is usual to normalize all attribute values to:

Answer 21

normalize attribute values to between 0-1.

Question 22

Normalizing Euclidean distance: symbolic attributes (non-numeric). The difference between two different values is usually expressed:

Answer 22

one (mismatch), zero (match)

Question 23

normalizing Euclidean distance formula - missing attributes are:

Answer 23

taken to be 1 (maximally different)

Question 24

normalizing Euclidean distance formula: For numeric attributes, the diff between two missing values is also taken as 1. However, if just one value is missing, the distance can be:

Answer 24

taken as (normalized) size of the other value X or 1-X, whichever is larger (as large as possibly)

Question 25

instance-based learning is slow because you are:

Answer 25

calculating distance from every member of the training set

Question 26

Nearest neighbors can be found using a:

Answer 26

kD-tree

Question 27

kD-Tree

Answer 27

Binary tree which divides input space with a hyperplane. k = no# of attributes.

Question 28

kD-trees: note that the hyperplanes are not \_

Answer 28

decision boundaries

Question 29

Kd-tree: choosing the median value may yield skinny hyperrectangles. Rather,

Answer 29

use the mean & point closest to that.

Question 30

Kd-tree: instance-based learning advantage; can update it incrementally. To do this for a kd-tree, determine which _ contains the new point and find its \_.

Answer 30

leaf node, hyperrectangle

Question 31

Kd-tree: corners of rectangular regions awkward? Use:

Answer 31

hyperspheres, rather than hyperrectangles

Question 32

kD-trees do not depend on regions being disjoint, the _ defines k-dimensional hyperspheres ("\_") that cover the data points, and \_

Answer 32

ball tree, balls, arranges them into a tree

Question 33

Ball Tree: regions can overlap, but points in the overlap are assigned to \_

Answer 33

only one of the overlapping balls

Question 34

Nearest-neighbor instance-based learning: k-nearest-neighbor: k of nearest neighbors: determine class using:

Answer 34

a majority vote

Question 35

kD Trees: worthwhile only when the attribute number is small:

Answer 35

up to 10

Question 36

Ball trees, an instance of a more general structure called: \_. Sophisticated algorithms can create trees that deal successfully with _ of dimensions.

Answer 36

a metric tree, thousands

Question 37

kD Trees: Instead of storing all training instances, you can \_.

Answer 37

compress into regions

Question 38

kD trees: discretizing numeric attributes into intervals (& for nominal "intervals" consisting of a single point). Determine which intervals the test-instance resides by voting. This is called:

Answer 38

voting feature intervals

Question 39

Distretizing into intervals, then determining which intervals test-instances reside, by voting.

Answer 39

voting feature intervals

Question 40

"Different ways in which the result of clustering can be expressed: _ (instances belong in one group). _ (instances belong in several groups). _ (instances belongs to groups with a probability), or _ (hierarchical)."

Answer 40

exclusive overlapping probabilistic hierachical

Question 41

The classic clustering technique is called \_

Answer 41

k-means

Question 42

K-means. What does the parameter k stand for?

Answer 42

how many clusters are being sought

Question 43

k-means - how are clusters chosen?

Answer 43

k-clusters are chosen at random, Euclidean distance metric

Question 44

k-means - all instances are assigned _ according to the \_

Answer 44

to one cluster center, Euclidean

Question 45

k-means: 'means' part: the _ of the instances in each cluster

Answer 45

Centroid or mean, these are the new center values for their respective clusters.

Question 46

k-means: Iteration continues until the same points k-means: Iteration continues until the same points are assigned to each cluster in:are assigned to each cluster in:

Answer 46

consecutive rounds

Question 47

k-means: choosing the cluster center to be the centroid _ from each of the cluster's points to its center.

Answer 47

minimizes the total sqrd distance

Question 48

k-means: to increase the chance of finding a global minimum people often\_ and choose the best final result—the one with the smallest \_.

Answer 48

run the algorithm several times with different initial choices, total sqrd distance

Question 49

k-means clustering can be dramatically improved by careful choice of the initial cluster centers, often called \_.

Answer 49

seeds

Question 50

k-means: Instead of beginning with an arbitrary set of seeds, can choose initial seed, then choose a second seed with a probability that is _ the distance from the first.

Answer 50

proportional to the square of

Question 51

k-means; seeding intelligently using the procedure, called _ improves both speed and accuracy over the original algorithm with random seeds.

Answer 51

k-means++

Question 52

1R for 1-rule

Answer 52

one-level decision tree, in the form of a set of rules, that test one attribute

Question 53

1R: deals with Missing Attributes in simple but effective ways. Missing is treated as

Answer 53

another attribute value

Question 54

1R: deals with Numeric Attributes in simple but effective ways. Use the \_

Answer 54

discretization

Question 55

1R -may form an excessively large number of categories (ID\_code) This phenomenon is known as \_

Answer 55

overfitting

Question 56

1R: to avoid overfitting when discretizing a numeric attribute, a minimim limit is imposed on

Answer 56

the no# of instances of the majority class in each partition.

Question 57

the gain ratio, provided that the information gain for that attribute is at least as great as the _ information gain for all the attributes examined.

Answer 57

maximizes, average

Question 58

Information gain is biased _ even though these won't work on new data

Answer 58

towards attributes with many values

Question 59

Use Gain ratio to deal with the problem with information gain, look at the \_

Answer 59

split entropy

Question 60

split entroy formula, relates the size of this subset relative to \_

Answer 60

the size of entire set

Question 61

gain ratio formula

Answer 61

Gain(S,A)/SplitEntropy(S,A)

Question 62

A series of improvements to ID3 culminated in a practical and influential system for decision tree induction called \_

Answer 62

c4.5

Question 63

covering algorithms: seek a way of covering all instances in a class & excluding all else - at each stage identifying a rule that covers some of the instances.

Answer 63

covering approach

Question 64

COVERING ALGORITHMS: CONSTRUCTING RULES lead to

Answer 64

Rules, not trees

Question 65

covering algorithms: replicated subtree problem

Answer 65

rules can be symetric whereas trees must select one attribute to split on first

Question 66

covering algorithms: A decision tree split takes all classes into account in trying to maximize the purity of the split, whereas

Answer 66

rules concentrates on one class at a time, disregarding other classes

Question 67

covering algorithms: accuracy/condidence

Answer 67

instances predicted correctly, as a proportion of the instances to which the rule applies

Question 68

covering algorithms: coverage also called

Answer 68

support

Question 69

covering algorithms: accuracy also called

Answer 69

confidence

Question 70

covering algorithms: In the event of a tie, we

Answer 70

choose the rule with the greater coverage

Question 71

coverage

Answer 71

covering algorithms: number of instances predicted correctly

Question 72

PRISM can be described as \_

Answer 72

separate-and-conquer algorithms

Question 73

PRISM method

Answer 73

Identify rule that covers many instances, separate out, continue on remaining instances

Question 74

the information value for info([2, 3]) is _ bits

Answer 74

−2/5 × log 2/5 − 3/5 × log 3/5

Question 75

u is the

Answer 75

mean

Question 76

Naïve Bayes: Word frequencies can be accommodated by applying a \_

Answer 76

modified form of Naïve Bayes called multinomial Naïve Bayes.

Question 77

Naïve Bayes: a document can be viewed as a \_

Answer 77

bag of words

Question 78

Naïve Bayes. If you suspect it isn't normal but don't know the actual distribution, there are procedures for "\_" that do not assume any particular distribution for the attribute values.

Answer 78

kernel density estimation

Question 79

information gain of the ID or Day of the week as attribute would be the same as

Answer 79

the information at the root

Question 80

gain ratio modification can overcompensate. One fix is to choose the attribute that \_, provided that the information gain for that attribute is at least as great as the _ for all the attributes examined.

Answer 80

that maximizes the gain ratio, average information gain

Question 81

information: amount of information obtained by making a decision. When the number of yes's and no's equal:

Answer 81

information reaches a maximum

Question 82

# choose best splitting attribute: information gain gain(temperature) = 0.571 bits gain(humidity) = 0.971 bits gain(windy) = 0.020 bits

Answer 82

humidty

Question 83

An important domain for machine learning is document classification, in which each instance _ and the instance's class is the\_

Answer 83

represents a document, document topic

Question 84

One of the really nice things about Naïve Bayes is that \_.

Answer 84

missing values are no problem

Question 85

This method goes by the name of _ because it's based on Bayes' rule and assumes independence —it is only valid to multiply probabilities when the events are \_.

Answer 85

naïve bayes, independent

Question 86

The gain ratio is derived by taking into account _ the dataset, disregarding any information about the class

Answer 86

the number and size of daughter nodes into which an attribute splits

Question 87

information is measured in:

Answer 87

units called bits