Practice Exam

Question 1

T/F
Q-learning can learn the optimal Q-function Q without ever
executing the optimal policy.

Answer 1

True

Yes, this is a property called off-policy learning.

Question 2

Which of the following would the best reward function for a robot
that is trying to learn to escape a maze quickly (assume a discount of $$\gamma = 1$$):

(A) Reward of +1 for escaping the maze and a reward of zero at all other times.
(B) Reward of +1 for escaping the maze and a reward -1 at all other times.
(C) Reward of +1000 for escaping the maze and a reward 1 at all other times.

Answer 2

(B) Reward of +1 for escaping the maze and a reward -1 at all other times.

Question 3

What does regret let us quantify?

(A) Whether our policy is optimal or not.
(B) The relative goodness of exploration procedures.
(C) The negative utility of a state like a fire pit.
(D) How accurately we estimated the probabilities of the transition function

Answer 3

(B) The relative goodness of exploration procedures.

Question 4

Which of the following is NOT True for both MDPs and Reinforcement Learning?

(A) A discounted future reward is used.
(B) An instantaneous reward is used.
(C) After selecting an action at a state, the resulting state is probabilistically determined.
(D) The values for the transition function are known in advance.

Answer 4

(D) The values for the transition function are known in advance.

Question 5

T/F
The utility function estimate must be completely accurate in order to get an optimal policy.

Answer 5

False

Question 6

What is a contraction?
(A) The time savings from estimating the optimal policy via policy iteration instead of value iteration.
(B) Part of the proof of convergence for the value iteration algorithm.
(C) A shorter path to a node in the A* algorithm when that node is already present on the priority queue.
(D) The part of the state space that is not observable in partially observable MDPs.

Answer 6

(B) Part of the proof of convergence for the value iteration algorithm.

Question 7

In the MDP framework we model the interaction between an agent and an environment. Which of the following statements are true of that framework

2.3.1
The agent selects actions, which deterministically move it to a new state in the environment.

Answer 7

False

Question 8

In the MDP framework we model the interaction between an agent and an environment. Which of the following statements are true of that framework

2.3.2
The agent receives a reward only once it arrives in its goal state.

Answer 8

False

Question 9

You roll two regular six-sided dice. What is the probability of getting a total sum of 10 or more given that the first dice shows a 6? Write as a decimal.

Answer 9

0.5

Question 10

How many ways are there to apply the chain rule to a joint distribution with $$N$$ random variables?

(A) $$N$$
(B) $$N^2$$
(C) $$2^N$$
(D) $$N!$$

Answer 10

(D) $$N!$$

Question 11

T/F
The Markov property says that given the past state, the present and the future are independent.

Answer 11

False

Question 12

If a process is stationary, it means that:

(A) the state itself does not change
(B) the conditional probability table does not change over time
(C) the transition table is deterministic
(D) the agent has reached a terminal state

Answer 12

(B) the conditional probability table does not change over time

Question 13

Which of the following is unnecessary to construct a dynamic Bayesian model (DBN)?

(A) The sensor model.
(B) The transition model.
(C) The prior distribution over the state variables.
(D) Multiple state and evidence variables.

Answer 13

(D) Multiple state and evidence variables.

Question 14

What is the effect of the Markov assumption in n-gram language models?

(A) It makes it possible to estimate the probabilities from data.
(B) Long distance relationships, like subject verb agreement, are taken into account.
(C) The probability of a word is determined by all previous words in the sentence.
(D) The probability of a word is determined only by a single preceding word.

Answer 14

(A) It makes it possible to estimate the probabilities from data.

Question 15

How are n-gram language models typically evaluated?

(A) Correlation with human judgments
(B) Cross-entropy measured against gold standard labels
(C) Perplexity on a test set
(D) Precision and recall

Answer 15

(C) Perplexity on a test set

Question 16

T/F
The conditional probability distribution of a variable in a Bayesian network should be specified based on the probability distributions of all of the other variables (nodes).

Answer 16

False

Just need the parents

Question 17

Write the joint probability for the Bayes’ Net shown below, encoding its independence assumptions into your equation. $$P(A, B, C, D, E) =$$

Answer 17

[$$P(A) \cdot P(B) \cdot P(C|A) \cdot P(D|C,A) \cdot
P(E|C,B)$$

Question 18

Which of the following is true of locally structured (sparse) systems?

(A) Each subcomponent must interact directly with all the other components.
(B) The structure grows linearly in complexity (rather than exponentially).
(C) Every variable cannot be influenced by all of the others.
(D) All such systems are compact.

Answer 18

(B) The structure grows linearly in complexity (rather than exponentially).

Question 19

T/F

It is possible that more than one Bayesian network can be used to represent the same joint distribution.

Answer 19

True

Question 20

If two variables (nodes) X and Y in a Bayesian network do not share a path, which of the following must be true?

(A) X and Y can never be true at the same time.
(B) X and Y are conditionally independent.
(C) X has a direct influence on Y.
(D) There exists a causal relationship between X and Y

Answer 20

(B) X and Y are conditionally independent.

Question 21

T/F

The Naive Bayes model is “naive” because it assumes that the features are conditionally independent of each other, given the class.

Answer 21

True

Question 22

Write down the form of the joint probability model $$P(X_1, X_2, X_3, Y )$$ for this data using the Naive Bayes assumption.

Answer 22

$$P(Y)P(X_1|Y)P(X_2|Y)P(X_3|Y)$$

Question 23

Which of the following can handle unknown contexts (assume words not in the vocabulary are all assigned to the same keyword)?
(Check all that apply.)

[A] Maximum Likelihood Estimate
[B] Stupid backoff
[C] Laplace Smoothing
[D] Smart backoff

Answer 23

[B] Stupid backoff
[C] Laplace Smoothing

Question 24

Which of the following best characterizes the difference between parametric and nonparametric models?

(A) Parametric models cannot summarize data with a large number of training examples.
(B) Parametric models can be used if each hypothesis considers all of the other training examples to make the next prediction.
(C) Instance-based learning and memory-based learning use parametric models.
(D A parametric model has a fixed size on the number of parameters.

Answer 24

(D A parametric model has a fixed size on the number of parameters.

Question 25

Which of the following is NOT true about the bag-of-words model?

(A) Each position is identically distributed.
(B) Predict label conditioned on feature variables.
(C) It is sensitive to word order or reordering.
(D) All positions share the same conditional probabilities.

Answer 25

(C) It is sensitive to word order or reordering.

Question 26

Which of the following are true about linear functions?

(A) It could be only used for classification.
(B) All sets of data points are linearly separable.
(C) When using the perceptron learning rule, the weights are updated when the actual output does not match the hypothesis output.
(D) The learning rule must be applied to one example at a time.

Answer 26

(C) When using the perceptron learning rule, the weights are updated when the actual output does not match the hypothesis output.

Question 27

T/F
For linearly separable data, there exists only one decision boundary that separates the classes.

Answer 27

False
raw out put > 0 -> 1, else 0

Question 28

T/F
If a binary perceptron has a high bias value, it is easier for the perceptron to output a 1.

Answer 28

True

Question 29

T/F
If all of its weights are greater than or equal to 0, a binary perceptron will always fire.

Answer 29

False

Question 30

Which of the following is correct about perceptrons?

(A) A perceptron is guaranteed to learn a decision boundary that perfectly separates the data within a finite number of training steps.
(B) A perceptron can only converge when the data points are linearly separable.
(C) The training error always decreases after each run on the entire training set.
(D) A perceptron can find multiple decision boundaries.

Answer 30

(D) A perceptron can find multiple decision boundaries.

[[A - only correct for linear separable data.]]
[[B - can converge with a decaying learning rate.]]
[[C - Typically, the variation across runs is very large.]]

Question 31

A perceptron converges to a minimum-error solution when:

(A) A fixed learning rate is used
(B) The learning rate increases as training time increases
(C) The learning rate decreases as training time increases
(D) The learning rate decreases only when validation error decreases

Answer 31

(C) The learning rate decreases as training time increases

Question 32

Relative to perceptrons, neural networks gain their power from

(A) smooth activation function
(B) stacking of layers
(C) convolution
(D) backpropagation

Answer 32

(B) stacking of layers

Question 33

In neural networks, nonlinear activation functions: Please check all that apply.

[ ] Make it possible to do the gradient calculation in
backpropagation, as opposed to using step function which isn’t differentiable.
[ ] Help to learn nonlinear decision boundaries.
[ ] Are applied only to the output units.
[ ] Always output values between 0 and 1.

Answer 33

[X] Make it possible to do the gradient calculation in
backpropagation, as opposed to using step function which isn’t differentiable.
[X] Help to learn nonlinear decision boundaries

Question 34

Which of the following is correct about RNN?

(A) RNN network uses an earlier layer’s outputs as inputs.
(B) The hidden layer of RNN includes a recurrent connection as part of its outputs.
(C) Forward inference in RNN means mapping a sequence of outputs
to a sequence of inputs.
(D) RNN is an effective approach for visual feature extraction.

Answer 34

(A) RNN network uses an earlier layer’s outputs as inputs.

Question 35

Which of the following are NOT taken into consideration by the backpropagation algorithm?

(A) The rate of change of the cost with respect to any weight.
(B) The rate of change of the cost with respect to any bias.
(C) The error of one layer in terms of the error in the next layer.
(D) The rate of change of one weight with respect to the other weights of neurons in the same layer.

Answer 35

(D) The rate of change of one weight with respect to the other weights of neurons in the same layer.

Question 36

Which of the following is NOT true about activation functions?

(A) Sigmoid function maps the output into the range [0, 1] and it is differentiable.
(B) The tanh function is a variant of the sigmoid function and maps outlier values toward the mean.
(C) The result of ReLU for negative numbers is the same as a linear function.

Answer 36

(C) The result of ReLU for negative numbers is the same as a linear function.