Chapter 2

Question 1

In a perceptron with two columns (layers) of perceptron-like simulated neurons (units), uses a 18 x 18 pixel image, a layer of three so-called hidden units, along with its layer of ten output units, each corresponding to one of the possible digit categories. How many inputs are there?

Answer 1

324 (18 × 18) inputs, each of which is set to the intensity value of the corresponding pixel in the input image.

Question 2

Describe the connections of a multilayered neural network

Answer 2

Each input has a weighted connection to each hidden unit, and each hidden unit has a weighted connection to each output unit.

Question 3

Why is a network known as multilayered?

Answer 3

because it has two layers of units (hidden and output) instead of just an output layer.

Question 4

What are deep networks?

Answer 4

In principle, a multilayer network can have multiple layers of hidden units; networks that have more than one layer of hidden units are called deep networks. The “depth” of a network is simply its number of hidden layers.

Question 5

How are the units similar and how do they differ in a multilayered neural network compared to a perceptron?

Answer 5

Similar to perceptrons, each unit here multiplies each of its inputs by the weight on that input’s connection and then sums the results. However, unlike in a perceptron, a unit here doesn’t simply “fire” or “not fire” (that is, produce 1 or 0) based on a threshold; instead, each unit uses its sum to compute a number between 0 and 1 that is called the unit’s “activation.” If the sum that a unit computes is low, the unit’s activation is close to 0; if the sum is high, the activation is close to 1.

Question 6

Describe how the multilayered algorithm would work in processing an image such as a handwritten 8

Answer 6

Each hidden unit computes its activation value; these activation values then become the inputs for the output units, which then compute their own activations. The activation of an output unit can be thought of as the network’s confidence that it is “seeing” the corresponding digit; the digit category with the highest confidence can be taken as the network’s answer—its classification.