MLPs

Question 1

Who came up with NETtalk?

Answer 1

Sejnowski & Rosenberg, 1987

Question 2

What does NETtalk do?

Answer 2

The task is to learn to pronounce English text from examples (text-to-speech)

Question 3

What is the training data for NETtalk?

Answer 3

a list of

Question 4

What are the inputs and outputs of NETtalk?

Answer 4

Input: 7 consecutive characters from written text presented in a
moving window that scans text

Output: phoneme code giving the pronunciation of the letter at the
center of the input window

Question 5

Describe the network topology of NETtalk

Answer 5

7x29 binary inputs (26 chars + punctuation marks),

80 hidden units and

26 output units (phoneme code).

Sigmoid units in hidden and output layer

Question 6

What is the activation function of the hidden and output layers of NETtalk?

Answer 6

Sigmoid units in hidden and output layer

Question 7

What is the accuracy of NETtalk after 50 epochs?

Answer 7

Training protocol: 95% accuracy on training set after 50 epochs
of training by full gradient descent. 78% accuracy on a test set

Question 8

Give example of another MLP similar to NETtalk

Answer 8

DEC-talk

Question 9

Give 3 examples of MLPs

Answer 9

NETtalk : converts text to speech (English)

ALVINN: Autonomous Land Vehicle In a Neural Network

Falcon: Frauds detection in credit card transactions

Question 10

Why is adding layers in multi-layer networks harmful?

Answer 10

– OVERFITTING: the increased number of weights quickly leads to data
overfitting (lack of generalization)

– SGD trap: huge number of (bad) local minima trap the gradient descent
algorithm

– Vanishing or exploding gradients (the update rule involves
products of many numbers) cause additional problems

Question 11

Why do we need deeper layers?

Answer 11

With one hidden layer, the number of required nodes and weights grows exponentially fast

The deeper the network the fewer nodes are required to model “complicated” functions