Chapter 3: Decision Making

Question 1

What is a knowledge base made out of?

Answer 1

Sentences

Question 2

What is the syntax of sentences?

Answer 2

Propositional/Predicate logic

Question 3

What is meaning of sentences?

Answer 3

Semantics

Question 4

What does M(s) denote

Answer 4

All models where the sentence s is true.

Question 5

What does a entails b mean?

Answer 5

This means that if a is true then b is true. In model terms M(b) is a subset of M(a).

Question 6

What is inference?

Answer 6

Inference is the reasoning process of deriving logical consequences of premise

Question 7

What is an inference algorithm that derives entailed sentances?

Answer 7

Sound or truth preserving

Question 8

Why is soundness highest priority?

Answer 8

An unsound inference would make up stuff as it goes to reach it’s conclusion (useless)

Question 9

When is an inference algorithm complete?

Answer 9

If it can derive any sentence that is entailed

Question 10

When is a sentence valid?

Answer 10

If it is true in all models

Question 11

When is a sentence satisfiable?

Answer 11

If it is true in some models

Question 12

What is a proof?

Answer 12

A chain of conclusions that lead to a desired goal

Question 13

What is Monotonicity?

Answer 13

Set of entailed sentences can only increase if more information is added into knowledge base

Question 14

Can Proof by Resolution always find an answer if it exists?

Answer 14

True

Question 15

What is Proof by contradicition?

Answer 15

A proof by resolution technique, which adds the opposite of what we are trying to prove into the KB and tries to obtain an empty clause (contradiction).

Question 16

What does Reinforcement learning attempt to achieve?

Answer 16

Many animals use rewards such as food in their environment as reward.

Reinforcement learning attempts to mimic this notion by making an agent that knows nothing about their environment achieve it’s goal.

Question 17

What is an optimal policy?

Answer 17

Actions agent can take that maximizes reward

Question 18

Methods of obtaining optimal policy

Answer 18

• Monte Carlo Method: Full look-ahead (explore all consecutive actions)
• Temporal Difference Method: One-step lookahead (Explore only one consecutive action)

Question 19

What is Markov Chain Rule?

Answer 19

Probability of reaching next state is only determined by current state

Question 20

What does Markov Decision Process Obey?

Answer 20

Markov chain rule

Question 21

What is Q-Learning?

Answer 21

Type of temporal difference reinforcement learning algorithm

Question 22

What does Q-Learning enable agents to do?

Answer 22

Act optimally in Markovian domains

Question 23

Does Q-Learning need to know it’s environment beforehand?

Answer 23

No

Question 24

What is Swarm Intelligence?

Answer 24

Multi-agent system inspired by insect colonies such as ants that work together.

Question 25

Does Ant Colony Optimization need to be changed for new problem domains?

Answer 25

False, but it could lead to a more optimal solution.

Question 26

Does Ant Colony Optimization need to be used in static environments?

Answer 26

False, can be used in scholastic time-varying environments)

Question 27

What can happen if agent always chooses action which maximizes Q-value?

Answer 27

Agent may pick non-optimal solution. Sticking to table is exploitation.