Class 6

Question 1

computational learning theory

Answer 1

lies at the intersection of AI, stats, and theoretical CS

Question 2

sample complexity

Answer 2

number of required examples to get to probably approximately correct

Question 3

approximately correct

Answer 3

a hypothesis that is consistent with low error rate after a large set of training examples

Question 4

linear functions

Answer 4

“fitting a straight line”

Question 5

linear regression

Answer 5

task of finding the best fitting line

Question 6

weight space

Answer 6

all of the possible settings for the weights

Question 7

alpha

Answer 7

step size, also called learning rate

Question 8

epoch

Answer 8

step that covers all the training examples

Question 9

decision boundary

Answer 9

line that separates two classes

Question 10

linear separator

Answer 10

linear decision boundary

Question 11

logistic regression

Answer 11

process of fitting the weights to a model to minimize loss

Question 12

parametric model

Answer 12

learning model that summarizes data with a set of parameters of fixed size (independent of the number of training examples)

Question 13

nonparametric model

Answer 13

learning model that cannot be characterized by a bounded set of parameters – this method retains all data points as part of the model

Question 14

table lookup

Answer 14

simplest instance based learning model – all training examples put into table, doesn’t generalize well

Question 15

curse of dimensionality

Answer 15

nearest neighbors works well in low dimensions with plenty of data – at higher dimensions it doesn’t work well

Question 16

locality sensitive hash

Answer 16

get around randomness and exact matching issues found in hash tables

Question 17

maximum margin separator

Answer 17

a decision boundary with the largest possible distance to example points – helps them generalize well

Question 18

kernel trick

Answer 18

SVMs can embed data into a higher dimensional space using this trick

Question 19

ensemble learning

Answer 19

selects a collection of hypotheses and combines their predictions by averaging, voting, or some other means of ML

Question 20

bagging

Answer 20

short for bootstrap aggregating

Question 21

random forest model

Answer 21

form of decision tree bagging in which extra steps to make ensemble of K trees more diverse to reduce variance

Question 22

boosting

Answer 22

most popular ensemble method, used a weighted training set, all examples start with equal weight, weight increased if training example fails, able to overcome any amount of bias in base model, approximates Bayesian learning

Question 23

gradient boosting

Answer 23

form of boosting that uses gradient descent as opposed to weighted examples

Question 24

semisupervised learning

Answer 24

typeof learning that involves giving a few labeled examples and using them to mine more information from a large collection of unlabeled examples

Question 25

crowdsourcing

Answer 25

using paid workers or unpaid volunteers operating over the internet