Introduction

Question 1

The true challenge to AI proved to be…

Answer 1

Solving tasks that are easy for people to perform, but hard for people to describe formally (intuitive, automatic tasks)

Question 2

Knowledge base approach to AI

Answer 2

AI approach that enables a computer to reason automatically about statements in a formal language using logical inference rules

Question 3

Machine Learning

Answer 3

The ability for AI systems to acquire their own knowledge by extracting patterns from raw data.

Question 4

Performance of simple ML algorithms (logistic regression/naive Bayes) depends heavily upon…

Answer 4

the representation of the data they are given

Question 5

Feature

Answer 5

A specific piece of information included in the representation of the data

Question 6

Many AI tasks can be solved by…

Answer 6

designing the right set of features to extract for that task, then providing these features to a simple ML algorithm

Question 7

Representation Learning

Answer 7

Uses ML to discover not only the mapping from representation to output but also the representation itself.

Question 8

When designing features or algorithms for learning features, our goal is usually…

Answer 8

to separate the factors of variation that explain the observed data

Question 9

Deep learning solves the difficulties in representation learning by…

Answer 9

introducing representations that are expressed in terms of other, simpler representations.

Question 10

Deep learning enables the computer to…

Answer 10

build complex concepts out of simpler concepts

Question 11

Another perspective on deep learning is…

Answer 11

the depth enables the computer to learn a multistep program. Each layer of representation is the state of the computer’s memory after executing another set of instructions in parallel. Later instructions can refer back to results of previous instructions. The state is more like pointers and counters - it has nothing to do with the input content, but helps the model organize its processing.

Question 12

2 main ways of measuring the depth of a model

Answer 12

1) The length of the longest path through a flow chart that describes how to compute each output given the model’s inputs 2) The depth of the graph describing how concepts are related to each other

Question 13

Consensus about how much depth a model requires to qualify as “deep”

Answer 13

There is none

Question 14

Deep learning is the study of…

Answer 14

models that involve a greater amount of composition of either learned functions or learned concepts than traditional machine learning does

Question 15

Deep learning is a particular kind of ML that…

Answer 15

achieves great power and flexibility by representing the world as a nested hierarchy of concepts, with each concept defined in relation to simpler concepts, and more abstract representations computed in terms of less abstract ones.

Question 16

3 waves of deep learning development

Answer 16

1) Cybernetics in the 1940s-1960s 2) Connectionism in the 1980s-1990s 3) Current resurgence as deep learning since 2006

Question 17

Artificial Neural Networks

Answer 17

One of the names that deep learning has gone by, emphasizing the perspective that models are engineered systems inspired by the biological brain. While the earliest algorithms were intended to model how learning happens in the brain, and neuroscience is still regarded as an important source of inspiration, it is no longer the predominant guide for the field (primarily because we do not understand enough about the brain).