ML

Question 1

tf: In tensorflow you imagine

Answer 1

everything you are computing as a graph
nodes are the transformations on the data, or functions you are running on the data. These can have multiple inputs and outputs.
The edges (the things connecting nodes) are the data.

Question 2

dl: back propogation is

Answer 2

looking at the output of a deep neural network model and comparing it to the desired output. Based on the difference between the correct answer and the prediction you adjust the layer right before to create the correct answer. Then based on the error in the second last layer, you adjust the third last layer and so on.

Question 3

ml: The points plotted near the decision boundary are called support vectors because

Answer 3

the are the ones that force the decision boundary to be where it is.

Question 4

ml: A Support Vector Machine is similar to a Nearest Neighbors because

Answer 4

An SVM only keeps the points that define the decision boundary, while NN keeps the points that do not influence the decision boundary as well as points that do.

Question 5

tf: A tensor is

Answer 5

a typed ndarray

Question 6

tf: A one hot vector is

Answer 6

a vector with zero in all columns besides one. The column that one is in represents the class it belongs to.

Question 7

ml: MNIST is a

Answer 7

computer vision dataset with images of handwritten digits and their labels

Question 8

ml: softmax is a

Answer 8

multinomial logistic regression

Question 9

ml: Softmax is good for

Answer 9

when you need the probabilities of a record belong to classes

Question 10

tf: A bias is used to

Answer 10

tell the algorithm that a certain class is more frequent in general

Question 11

tf: To create the table that hold all your samples, type

Answer 11

x = tf.placeholder(tf.float32, [None, 784])

Question 12

tf: in x = tf.placeholder(tf.float32, [None, 1000]), None means

Answer 12

that that dimension can vary

Question 13

tf: in x = tf.placeholder(tf.float32, [None, 784]), 1000 is

Answer 13

The number of columns

Question 14

tf: To create the weights variable, type

Answer 14

W = tf.Variable(tf.zeros([1000, 10]))

Question 15

tf: To create the biases variable, type

Answer 15

b = tf.Variable(tf.zeros([10]))

Question 16

tf: To create a softmax model, type

Answer 16

y = tf.nn.softmax(tf.matmul(x, W) + b)

Question 17

tf: a good cost function is called

Answer 17

cross-entropy

Question 18

tf: Gradient descent is a simple procedure, where

Answer 18

TensorFlow shifts each variable a little bit in the direction that reduces the cost

Question 19

ml: Using small batches of random data for taining is called

Answer 19

stochastic training

Question 20

tf: To create your cross entropy cost function, type

Answer 20

y_ = tf.placeholder(tf.float32, [None, 10])
cross_entropy = -tf.reduce_sum(y_*tf.log(y))

Question 21

tf: To initialize your gradient descent optimizer with a learning rate of 0.01 and a cost function called cross entropy, type

Answer 21

train_step = tf.train.GradientDescentOptimizer(0.01).minimize(cross_entropy)

Question 22

tf: To initialize all the variables and then run a session type

Answer 22

init = tf.initialize_all_variables()

sess = tf.Session()
sess.run(init)

Question 23

tf: a feed dict is

Answer 23

a dict that maps a var full of samples to the x and a var full of labels to the y

Question 24

tf: To connect to tensorflows c++ back end you use

Answer 24

a session

Question 25

ml: A bias

Answer 25

ads a number to the input times the weight

Question 26

ml: An activation function is

Answer 26

a function that takes is all of the inputs and then outputs a value

Question 27

ml: ReLU stands for

Answer 27

Rectified linear units

Question 28

ml: The rectifier functions formula is

Answer 28

f(x) = max(0, x)

Question 29

ml: The rectifier is

Answer 29

the most popular activation function for DNNs

Question 30

ml: A Convolutional Neural network is

Answer 30

a neural network structured in a way that is better for images.

Question 31

tf: The basic procedure for creating a tf model is

Answer 31

import the data
create the tensors
create a session
create your softmax layer
create your loss function
create the train step
evaluate

Question 32

tf: A more sophisticated optimizer than GradientBoostingOptimizer is

Answer 32

AdamOptimizer

Question 33

tf: keep_prob in the feed_dict

Answer 33

controls the dropout rate

Question 34

ml: A linear function is just a

Answer 34

giant matrix multiply

Question 35

ml: A Logistic Classifier is

Answer 35

a linear classifier

Question 36

tf: A softmax function takes all the scores from the linear functions and

Answer 36

turns them into class probabilities that together add up to one

Question 37

ml: Scores in the context of logistic classifiers are also called

Answer 37

logits

Question 38

ml: When you multiply to increase the size of your logits

Answer 38

the classifier makes the confident logits grow very quickly and becomes very confident

Question 39

ml: When you divide to decrease the size of your logits

Answer 39

the classifier makes the logitsmove closer together and becomes less confident

Question 40

ml: For one hot encoding

Answer 40

you make a vector with the same number of items as there are classes and then give each class one index of the array that represents it by making it’s value 1 while the rest of the values are 0.

Question 41

ml: A vector is

Answer 41

an array

Question 42

ml: To make one hot encoding more efficient for models with thousands of classes we use

Answer 42

imbeddings

Question 43

ml: Cross entropy is

Answer 43

the difference between the array of probabilities compared to your one hot encoded vector for the correct class.

Question 44

ml: Multinomial logistic classification

Answer 44

inputs use a linear function to produce logits. Logits go into a softmax function to create probabilities out of 1, and the probabilities are then compared to the one hot encoded vector using cross entropy.