00 Summaries

Question 1

Reinforcement learning

Answer 1

Reinforcement learning: How an agent can become proficient in an unknown environment, given only its percepts and occasional rewards

Utilities can be learned using:

• Direct utility estimation
• Adaptive dynamic programming (ADP)
• Temporal-difference (TD)

Three overall agent designs:

• Model-based design using a model and a utility function
• Model-free design, using an action-utility function Q
  
  • Q-unfctions can be learning by ADP or TD
• Reflex design, using a policy π

Value of information is relevant for active learning

Approximate functions are required for large state spaces

Question 2

Natural Language Processing

Answer 2

Natural Language Understatning
- Require empirical investigation of actual human behavior

Formal language theory
Phrase structure grammars
(Lexicalized) Probabilistic Context-free grammar (PCFG) formalism
Treebank
Augmented grammar for semantic interpretation
Parsing
Machine translation

Question 3

Learning from Examples

Answer 3

Learning from examples: Inductive learning of functions from examples. Inductive learning involves finding a hypothesis that agrees well with the examples.

Ockham’s razor (choose the simples consistent hypothesis)
Supervised learning
Classification (discrete-valued function)
Regression (continuous function)
Decision trees (with information-gain heuristic)
Perceptron
- Trained by simple weight update rule
Neural networks
- Represent complex nonlinear functions with a network of linear-treshold units.
- Multilayer feed-forward neural networks can represent any function
- Back-propagation algorithm implements a gradient descent in paramater space to minimize the output error

Question 4

Making Simple Decisions

Answer 4

Decision theory (what should an agent do) = Probability theory (what should an agent believe) + Utility theory (what does an agent want)

MEU
Decision networks (extension of Bayesian networks)
Value of information: Expected improvement in utility compared with making a decision without the information
Decision-theoretic expert systems

Question 5

Probabilistic Reasoning

Answer 5

Probabilistic Reasoning: Bayesian networks

- a representation for uncertain knowledge and conditional independence relationships

Question 6

Quantifying Uncertainty

Answer 6

Uncertainty arise because of laziness and ignorance
Probabilities express the agent’s inability to reach a definite decision regarding the truth of a sentence
Decision tehory
Prior probabilities and conditional probabilities
Full joint porobability distribution
Bayes’ rule
Conditional independence
Naive Bayes

Question 7

Inference in FOL

Answer 7

• Inference using rules (UI, EI) to propositionalize the problem
• Unification
  Generalized Modus Ponens (applied by forward- and backward-chaining)
  Resolution provides a complete proof system