Networks

Question 1

What is a binary neuron

Answer 1

the possible outcomes are just 0 and 1

Question 2

How to determine if a neuron is linearly separable

Answer 2

To determine if a neuron is linearly separable a decision boundary must be able to be drawn on the input space to seperate the positive outputs from the negative outputs.
This is done by making the motor function = to 0 and rearranging it so that it is in the form y=mx+c (a bias term can be intorduced i.e. w3 as c to represent the y intercept so that any line in the input space can be drawn). S1 = y s2= x and m and c are represented by the weights/ wires of the vehicle. If the positive and negative values cant be seperated by a straight line then the vehicle is non linearly separable.

Question 3

All linearly separable problems:

Answer 3

AND, OR, NAND- there are 16 different possible combinations.

Question 4

What is the AND problem

Answer 4

The AND neuron should generate an output only when both the first input and second input = 1. All other possible combinations do not produce an output.

Question 5

What is the OR problem

Answer 5

the OR neuron should generate an output when s1 or s2 = an input of 1 (and when both do) but not when neither are.

Question 6

What is the NAND problem

Answer 6

The NAND neuron should generate an output only doesnt produce a response when both the first input and second input = 1. All other possible combinations do produce an output. (opposite of AND).

Question 7

What is the XOR problem

Answer 7

Exclusive OR (XOR) is one of sixteen possible logical operations that can be used to translate two binary (1 or 0) inputs into a binary output.
An XOR circuit should generate an output of 1 when either the first input or the second is 1, and it should generate a 0 output if both inputs are 1 or if both inputs are 0.
It is not possible to draw a single line in the input space to separate combinations of inputs that should generate a 1 output from those that should generate a 0 output. XOR is therefore not a linearly separable problem.
Therefore it is impossible to define a set of wires for a vehicle (or equivalently a set of synaptic weights for a neuron) to implement XOR. This reveals a fundamental limitation about individual neurons, and motivates the use of networks of interconnected neurons, which can solve the XOR problem (and its logical complement).

Question 8

How to solve the XOR problem

Answer 8

Because it is no linearly separable, the problem requires networks of neurons.
NAND + OR + AND. Feedforward network.

Question 9

What is backpropogation?

Answer 9

With single neurons/ vehicles it is possible to use gradient descent to get the delta rule which allows us to reduce the error and to update the synaptic weights .
But with neural networks an extra step is required, more applications of the chain rule are required.