Reinforcement Learning

Question 1

Was sind Agents?

Answer 1

Question 2

Wann ist ein Agent autonomous?

Answer 2

Question 2

Was sind Rational Agents?

Answer 2

Question 3

Was sind Reflexive Agents?

Answer 3

Question 4

Was sind Agents with internal state?

Answer 4

Question 5

Was sind Goal-based agents?

Answer 5

Question 6

Was sind Agents with some use function?

Answer 6

Question 7

Beschreib den Markov Decision Process

Answer 7

Question 8

Was besagt die Markov Property?

Answer 8

Not dependent on history

Question 9

Was ist epsilon-greedy?

Answer 9

Question 10

Wie wirken sich die Wahl von epsilon und beta auf epsilon-greedy aus?

Answer 10

Question 11

Was ist eine q-table?

Answer 11

One-hot state encodings x One-hot action encodings

Question 12

Wie funktioniert tabular rl?

Answer 12

Question 13

Wie kann man Tabular RL mit q-tables als Deep RL realisieren?

Answer 13

Question 14

Deep RL

Wofür ist das implicit model of action selection?

Answer 14

Question 15

Wie funktioniert Temporal Difference (TD) Learning?

Answer 15

Question 16

Nenn die Bellman equation

Answer 16

Question 17

TD-learning advises to adapt the Q-value for the current (s,a). How?

Answer 17

Question 18

Beschreib den SARSA Algorithmus

Answer 18

Question 19

Vergleiche SARSA und Q-Learning

Answer 19

Question 20

Erkläre Actor-critic Learning

Answer 20

Question 21

Erkläre Deep RL für SARSA or Q-Learning.
Wie backpropagatet man?

Answer 21

Question 22

Was ist value-based RL?

Answer 22

Question 23

Was ist policy-gradient based RL?

Answer 23

maximize the expected future return R

Question 24

Wie schafft man es, dass bei Policy Gradient based RL alle Gewichte trainiert werden können?

Answer 24

Question 25

Was ist goal-conditioned RL?

Answer 25

Question 26

Was ist Experience Replay?

Answer 26

Performing many trials in the environment can be costly.
Solution: learn multiple times from the experience:

Question 27

Was ist Hindsight Experience Replay (HER)?

Answer 27

Agent knows how to get to any experienced state

Question 28

Was ist Hierarchical RL?

Answer 28

higher level tells lower level on which goal to perform

Question 29

Wie funktioniert Model-based RL? Wie bestimmt man eine Aktion?

Answer 29

like a tree search

Question 30

Was sind Limitations von Model-based RL?

Answer 30

exponential number of states/actions; cycles

Question 31

Erkläre: Advantage of Model-based vs Model-free RL

Answer 31

Question 32

Beschreib die Architektur von MuZero

Answer 32

Question 33

Beschreibe MuZero: Planning by Monte-Carlo Tree Search

Answer 33

Question 34

Wie wird MuZero trainiert?

Answer 34

via Experience Replay

Question 35

Wie kann man die MuZero Architektur erweitern? Was bringt das?

Answer 35

Question 36

Wie kann man MuZero im Bezug auf continous Actions erweitern?

Answer 36

Question 37

Which of the following statements on Hierarchical Reinforcement Learning are correct?
1. If the subtasks don’t yield rewards individually, Hierarchical RL can’t be used.
2. Hierarchical Agents always use the same set of actions on each level of the hierarchy.
3. In Hierarchical RL high-level agents set the goals for lower-level agents.

Answer 37

3

Question 38

Assume we have the following information on 2-itemsets generated with the Apriori algorithm:
Frequent: {𝐴,𝐵}, {𝐴,𝐶}

Not frequent: {𝐴,𝐷}

Which 3-itemsets could potentially be tested against in the following iteration of the Apriori algorithm, independent of any additional information on other itemsets?

1. {𝐴,𝐵,𝐷}
2. {𝐴,𝐶,𝐷}
3. {𝐵,𝐶,𝐷}
4. {𝐴,𝐵,𝐶}

Answer 38

4

Question 39

Answer 39

0,2/0,4=0,5

Question 40

Which of the following statements on Policy Gradient methods and the REINFORCE algorithm are correct?
1. The REINFORCE algorithm learns how to estimate the best Q-Values.
2. Policy Gradient-based RL learns to estimate the probabilities of the actions for a given state directly.
3. The REINFORCE algorithm uses the TD-error to update the network weights.

Answer 40

2

Question 41

Which of the statements about the k-Nearest Neighbors (k-NN) algorithm are correct?
1. k-NN can also be used for regression.
2. k-NN can be used for imputing missing values of both categorical and continuous variables.
3. k-NN performs much better if all of the data have the same scale.
4. k-NN is only defined for Euclidean distance metric.

Answer 41

1, 2 und 3

Question 42

The goal of model-based RL strategies is to reduce the complexity of searching for the best solution, in an environment in which the dynamics are known.
Stimmt das?

Answer 42

Ja

Question 43

Which of the following statements on the MuZero algorithm are correct?
1. In MuZero, a model of the world has to be provided.
2. MuZero uses planning in the latent space to find the best action.
3. For each episode, only one search tree has to be created.
4. The dynamics function maps a hidden state and an action to another hidden state.

Answer 43

2 und 4

Question 44

One of the major disadvantages of the Apriori algorithm is high computational complexity. Which of the following factors are predominantly responsible for this?
1. A high number of items in each itemset
2. A low minimum confidence
3. A high minimum support
4. A high number of itemsets in the database

Answer 44

1 und 4

Question 45

Answer 45

2/5=0,4

Question 46

In Hindsight Experience Replay the reward scheme is already known while the experiences are gathered.
Stimmt das?

Answer 46

Nein