Neuron Networks

Question 1

2014 1.a Black box(how to acquire knowledge used to make decisions, how knowledge stored)?

Answer 1

It means that only its input and output information is visible.
The neuron network are arranged in layers. The input to the net in layer 0 is passed layer by layer, each layer’s neuron units perform some computation before handling to the next layer. By adaptive interconnects (weights), the net learn to perform a set of mappings from input vectors to output vectors.

Question 2

Why black box feature pose problem?

Answer 2

Because if we can’t see what the inside of the neuron network, we can’t find the entity to take the responsibility for doing something wrong. For example, the financial forecasting, if we loses a lot of money, because the neuron network is a black box, we can’t find what was wrong inside.

Question 3

Example suitable using neuron network.

Answer 3

Optimising routing in communication networks.

Question 4

2014 1.b Limitation of perceptrons, need additional computational layers?

Answer 4

The only function that can be represented by a single layer net are those for which A and B class input vectors can be separated by a single line, namely linear separable. For those problems that are not linear separable, additional computational abilities is needed.
Take the XOR problem for example, truth table, pattern. It’s clear that no single line can be drawn to separate the two classes.

Question 5

2014 1.c How Rosenblatt group overcome lack of training hidden layers? Valid form of training?

Answer 5

They used a trial-and-error training method that involved a ‘preprocessing layer’.
No, it’s not a valid form of training, because the weights are acquired by random search, instead of something more efficient.

Question 6

2014 1.d Single layer perceptron one advantage over the multilayer perceptron? Why multilayer perceptron training can’t have.

Answer 6

The single layer perceptron training process will always eventually converge to a solution which correctly partitions the pattern space, but only if such a partition is theoretically possible. The MLP does not come with a guarantee of success, because it’s prevented by a number of factors and may be trapped in a local minimum.

Question 7

2014 1.e Hidden units, algebraic expression,weight factor and threshold?

Answer 7

Hidden units: 1的数量
Algebraic expression: y1= y2=
W = 1 or -1
S = n - 1/2 - number of input where f(x)=反x

Question 8

2014 2.a i) fixed point of the system?

Answer 8

The fixed point of the system is a stable state of the network, in which the recirculated output no longer changes the network state.

Question 9

2014 2.a ii) if fixed point the output, what is the input?

Answer 9

The initial state is the input, and it is the initial state that determines which fixed point(final state) it will reach.

Question 10

2014 2.b Why lowest energy state always fixed point?

Answer 10

Because the Hopfield update rule does not allow you to go anywhere else, once you are in the lowest energy state.

Question 11

2014 2.c Basin of attraction?

Answer 11

It’s a range of input states that will ultimately iterate to a local minimum. These local minima correspond to the stable states of the system.

Question 12

2014 2.d Zero weights and threshold, no useful CAM

Answer 12

If weights and thresholds are set to zero, in a iteration, nothing will change. This means that nothing is in the base of attraction of a local minimum. Therefore, the net doesn’t have any useful CAM properties.

Question 13

2014 2. e i)weight setting rule, give Hopfield function?

Answer 13

Weight setting rule
Threshold setting rule
Hopfield function

Question 14

2014 2. e ii) Why 0 threshold not surprised?

Answer 14

Because the two binary pattens (1, 0, 1), (0, 1, 0) are complemental.

Question 15

2014 2. e iii) State Transition Diagram?

Answer 15

1. 写出W, S=-w, H=-wxx+sx
2. 根据H计算Energy level
3. 根据W, S，以及firing rule, 写出Neuron update rules x=h(wx-s, x)
4. 根据update rule写出各个pattern转换的结果

Question 16

2014 2. e iv) Comment on the CAM properties of the system.

Answer 16

Ensure local minima of H corresponds to the pattern we want to store.

Question 17

2013 1.a Location and distributed representation of information. Advantage and disadvantages?

Answer 17

(one to one)In local representation, each neuron represents one concept. (many to many)While in the distributed representation, each concept is represented by many neurons and each neuron participates in the representation of many concepts.
Local: advantage: very easy to access the contents disadvantage: it may be quite vulnerable, because when you damage part of a hardware, then you lost everything.
Distributed: advantage: is more robust, disadvantage: very difficult to decode, because the knowledge is spread around.

Question 18

2013 1.b i) Why constructive BDN not feasible, learning procedure preferred?

Answer 18

Firstly, not all problems are binary. Secondly, the procedure for constructing the nets according to the rules is very costly, and it only gives a solution, not necessarily the most efficient solution.

Question 19

2013 1.c How MLP overcome limitations of BDN.

Answer 19

It has a efficient weight learning rule and produces non-binary outputs.
With a continuous and differentiable output function like sigmoid, it is possible to discover how small changes to the weights in the hidden layers affect the behaviour of the units in the output layer, and to update the weights of all neurons.

Question 20

2013 1.d i) Sigmoid function?

Answer 20

f(a)= 1/(1+e^-a) f’(a)= e^(-a)/(1+e^-a)^2

Question 21

2013 ii) Why not large initial weight values?

Answer 21

Large or very similar starting value impede learning. It causes the net not to follow desired gradient descent path.

Question 22

2013 iii) MLP weight updating rule, long time training.

Answer 22

Error contributed by neuron in layer i is proportional to the sum of the errors associated with the units in layer i+1, which dramatically increases training time.

Question 23

2013 2.a Hopfield ensures delta H never be positive.

Answer 23

if [ ] = 0, no change in Hi
if [ ] >0, Hi =0, switch on or leave on.
if [ ]

Question 24

2013 2.b i) Hopfield capacity? How it’s been used.

Answer 24

0.15N. Because it’s disappointedly low compared to 2^N binary patterns that can be represented, it hasn’t been widely used in the type of CAM applications. Though small modification can be made to improve CAM properties for smaller scale problems.

Question 25

ii) Undesirable side effects, and threshold setting rule.

Answer 25

If (x1…xn) is an assigned stable state, then its complement is also a fixed point. This symmetry is undesirable because it wastes the net’s limited storage capacity.
Si = -Wi0

Question 26

2013 2.c Multiple minima of Hopfield energy function desirable while multiple minima of a root mean squared error function as considered undesirable.

Answer 26

In the Hopfield function, there’s the energy surface, the minima corresponds to the stable state of the system. It’s where patterns are stored. While in the error function, it’s the error surface. Multiple minima gives rises to higher possibility of trapping in a local minimum.