Chapter 5

Question 1

AdaBoost

Answer 1

a boosting ensemble method algorithm

Question 2

artificial neural network (ANN)

Answer 2

Computer technology that attempts to build computers that operate like a human brain. The machines possess simultaneous memory storage and work with ambiguous information.

Question 3

attrition

Answer 3

loss of personnel, e.g. students, customers, staff, etc

Question 4

axon

Answer 4

An outgoing connection (i.e., terminal) from a biological neuron

Question 5

backpropagation

Answer 5

The best-known learning algorithm in neural computing where the learning is done by comparing computed outputs to desired outputs of training cases

Question 6

bagging

Answer 6

The simplest and most common type of ensemble method; it builds multiple prediction models (e.g., decision trees) from bootstrapped/resampled data and combines the predicted values through averaging or voting

Question 7

Bayesian network (BN)

Answer 7

these are powerful tools for representing dependency structure among variables in a graphical, explicit, and intuitive way

Question 8

Bayes theorem

Answer 8

this is a mathematical formula for determining conditional probabilities

Question 9

boosting

Answer 9

This is an ensemble method where a series of prediction models are built progressively to improve the predictive performance of the cases/samples incorrectly predicted by the previous ones

Question 10

conditional probability

Answer 10

the probability of event A given that an event B is known to have occured.

Question 11

cross-validation

Answer 11

involves randomly splitting the data into multiple groups such that N groups are used for training and M groups are used for testing and validation

Question 12

dendrites

Answer 12

The part of a biological neuron that provides inputs to the cell

Question 13

distance metric

Answer 13

A method used to calculate the closeness between pairs of items in most cluster analysis methods

Question 14

Euclidean distance

Answer 14

shortest path between two points

Question 15

heterogenous ensemble

Answer 15

These combine the outcomes of two or more different types of models such as decision trees, artificial neural networks, logistic regression, support vector machines, and others

Question 16

hidden layer

Answer 16

The middle layer of an artificial neural network that has three or more layers

Question 17

Hopfield network

Answer 17

a neural network architecture

Question 18

hyperplane

Answer 18

A geometric concept commonly used to describe the separation surface between different classes of things within a multidimensional space.

Question 19

information fusion

Answer 19

(or simply, fusion) A type of heterogeneous model ensembles that combines different types of prediction models using a weighted average, where the weights are determined from the individual models’ predictive accuracies

Question 20

k-fold cross-validation

Answer 20

A popular accuracy assessment technique for prediction models where the complete data set is randomly split into k mutually exclusive subsets of approximately equal size. The classification model is trained and tested k times. Each time it is trained on all but one fold and then tested on the remaining single fold. The cross-validation estimate of the overall accuracy of a model is calculated by simply averaging the k individual accuracy measures

Question 21

k-nearest neighbor (kNN)

Answer 21

A prediction method for classification as well as regression-type prediction problems where the prediction is made based on the similarity tok neighbors

Question 22

kernel trick

Answer 22

In machine learning, a method for using a linear classifier algorithm to solve a nonlinear problem by mapping the original nonlinear observations onto a higher-dimensional space, where the linear classifier is subsequently used; this makes a linear classification in the new space equivalent to a nonlinear classification in the original space

Question 23

Kohonen’s self-organizing feature map

Answer 23

A type of neural network model for machine learning

Question 24

Manhattan distance

Answer 24

the rectilinear distance between two points (sum of 2 shortest paths of a triangle)

Question 25

maximum margin

Answer 25

In machine learning the margin of a single data point is defined to be the distance from the data point to a decision boundary.

Question 26

Minkowski distance

Answer 26

a generalized distance formula that can be specified to get rectilinear or euclidean distance

Question 27

multi-layer perceptron

Answer 27

a feed-forward neural network architecture

Question 28

Naive Bayes

Answer 28

A simple probability-based classification method derived from the well-known Bayes’ theorem. It is one of the machine-learning techniques applicable to classification-type prediction problems

Question 29

neural computing

Answer 29

An experimental computer design aimed at building intelligent computers that operate in a manner modeled on the functioning of the human brain. See artificial neural network (ANN)

Question 30

neuron

Answer 30

A cell (i.e., processing element) of a biological or artificial neural network

Question 31

nucleus

Answer 31

The central processing portion of a neuron

Question 32

pattern recognition

Answer 32

A technique of matching an external pattern to a pattern stored in a computer’s memory (i.e., the process of classifying data into predetermined categories). Pattern recognition is used in inference engines, image processing, neural computing, and speech recognition

Question 33

perceptron

Answer 33

An early neural network structure that uses no hidden layer

Question 34

processing element (PE)

Answer 34

A neuron in a neural network.

Question 35

radial basis function (RBF)

Answer 35

generally speaking, Radial Basis Function (RBF) is a reasonable first choice for the kernel type. The RBF kernel aims to nonlinearly map data into a higher dimensional space; by doing so (unlike with a linear kernel), it handles the cases in which the relation between input and output vectors is highly nonlinear

Question 36

random forest

Answer 36

First introduced by Breiman (2000) as a modification to the simple bagging algorithm, it uses bootstrapped samples of data and a randomly selected subset of variables to build a number of decision trees, and then combines their output via the simple voting

Question 37

retention

Answer 37

opposite of attrition

Question 38

stacking

Answer 38

(a.k.a. stacked generalization or super learner) A part of heterogeneous ensemble methods where a two-step modeling process is used—first the individual prediction models of different types are built and then a meta-model (a model of the individual models) is built

Question 39

supervised learning

Answer 39

A method of training artificial neural networks in which sample cases are shown to the network as input, and the weights are adjusted to minimize the error in the outputs.

Question 40

stochastic gradient boosting

Answer 40

a boosting method that is gaining popularity due to it’s superior performance

Question 41

synapse

Answer 41

The connection (where the weights are) between processing elements in a neural network

Question 42

transformation (transfer) function

Answer 42

In a neural network, the function that sums and transforms inputs before a neuron fires. It shows the relationship between the internal activation level and the output of a neuron

Question 43

what-if scenario

Answer 43

It is an experimental process that helps determine what will happen to the solution/output if an input variable, an assumption, or a parameter value is changed