Computer Practicals Flashcards
In the lecture, you saw the following equation:
ai=f(∑ wij aj−bi)
Which aspects of the biological neuron are being modelled by the variables?
wij : Weight, so the strength of the synapses j to neuron i
bi: Bias, a value which determines how quickly a neuron will respond to particular inputs.
f: Transfer function, so the transduction of the incoming input to the output of the neuron.
ai: Output of the neuron i, in firing rate.
Which physiological aspects of real (biological) neurons are missing in this model?
Several aspects of real neurons are missing in this model, because this model is a gross oversimplification. Some examples:
Timing of spikes, there is no time component in this model.
Abundancy of dendrites / axon branches and their spatial arrangement.
Refractory period.
Adaptation, plasticity.
No biomolecular effects such as aging, hormones, disease etc.
A biologically plausible transfer function; a hardlimit transfer function is not possible in real life.
Effects of LTP.
Consider a neuron, which receives input from a single presynaptic neuron. We will now drop the subscripts and call the output of the neuron of interest a, the output of the presynaptic neuron p, and the bias b . consider two settings, namely w=1 and w=−1, and b=0.5 with a hard-limit function (hardlim) to start with.
Describe how output a varies as a function of p (where p is between 0 and 1), for both cases w=1 and w=−1.
w=1: This neuron simply multiplied the input p by 1, thus doesn’t change it. If this reached threshold b=0.5 it fires, i.e. the output a=1
w=−1: This neuron received negative input from the negative w, so this can be seen as inhibitory input. With p between 0 and 1, it will never reach threshold and spike (i.e. never reach a=1)
Assume now that the input gets a larger dynamic range and can vary between -4 and 4.
Describe again how the output a varies as a function of p .
When pp is allowed to vary from -4 to 4:
w=1: As before, the in input pp is multiplied by 1, thus stays the same. Ergo, when p reached threshold b=0.5 the cell will spike, as in a=1.
w=−1 : The input is multiplied by w=−1. In this case, a negative input will be able to reach threshold, i.e. p< −0.5. Other inputs will not reach threshold.
We further assume that we receive a binary signal as input (p is either 0 or 1), however now we can vary the bias, b (-2,2).We again study both cases w=1 and w=−1 . For what ranges of the bias b will our neuron be able to discriminate a binary input (which takes on either 1 or 0) coming from the pre-synpatic cell?
w=1: Here the best classification between 1 and 0 is the middle, i.e. b=0.5. As the weight won’t affect the p value, inputs of 0 will be < b=0.5 and inputs of 1 > b, giving appropriate outputs. If the input will only be 0 or 1, b should be in the range 0≤b<1
w=−1w=−1: When the input is multiplied by −1−1, it’s basically the same but negative, ergo bb should be in the range −1≤b<0.
What is the output of the hardlim function?
0 or 1
Give two alternative functions given in the first assignment
Linear function: purelin(x): f(x) = x
Hyperbolic tangent: tanh(x): f(x) = e^2x - 1 / e^2x + 1
What are the codomains of these two functions?
The codomain of the linear function is (−∞,∞), while for the hyperbolic tangent it is (−1,1).
Describe the graphs given by these two functions
Linear function: Gives a linear graph with a continuous output
Hyperbolic tangent: Gives an S shaped curve between -1 and 1 on y axis
What effect does the bias have on these two graphs
Linear function: Shifts the line along the x axis
Hyperbolic tangent: Shifts the line along the x axis
Does not change the shape of the function, just the amount required for the same output, higher bias means more shifted to the right and more input required for a higher output
Which configuration of transfer function and parameters classifies values of the input pp most strictly into ‘categories’? We refer to transfer functions that make such a strong distinction between two ranges of input values as classifiers.
To make a strong classifier, the hardlimit transfer function would be ideal. The values of b and w will depend on what you want to classify, but the w can’t be 0. Dependent on your input p, your threshold should be of a logical value, i.e. inputs and weight of 1 and a threshold of 500 will not classify.
For instance, as seen in previous questions, classifying binary inputs to binary outputs (a really boring classification), can be readily done with 0≤b<1, and a w=1
Conversely, which configuration is the worst at grouping values of the input in separate categories?
A linear transfer function (like purelin) will not give strict classifications, but a continuous output.
What could be the advantage of a poorly classifying configuration?
This will be an estimator. If you want your output to reflect the input in a continuous, linear fashion, you would need this configuration.
Which transfer function is the most biological one? Explain!
Of our transfer functions the hyperbolic tangend (tanh) function is the most biologically plausible one. These shapes of curves are often seen in biology and psychology
In which input range does the hyperbolic tangend (tanh) transfer function behave as an estimator of the input, and in which range as a classifier?
Estimator range: The middle of the tansig curve is relatively straight, and acts as an estimator, much like a linear transfer function.
Classifier range: The ends of the tansig curve plateau towards 0 and 1, and this acts similar to the hard limit transfer function.
What is the essential mathematical feature of a transfer function aimed at classification?
The feature of non-linearity; it needs a relatively strict distinction between values, such as we see in the hard limit function. This is highly non-linear.
Which aspects of how neurons render the transfer from synaptic input to output can be modelled with a sigmoid function?
- The sigmoid function has a threshold-like point, it has to reach certain input to start responding. 2. Moreover, in the middle, a small increase in input gives a large increase in output (i.e. it has a dynamic range). 3. furthermore, the function also has a maximum, where the output doesn’t increase further.
Which physiological aspects of real neurons can be seen as the counterpart of the bias in computational neurons?
This can be thought of as the resting potential of a neurons, spontaneous input of other neurons on said neuron, biomolecular effects (i.e. hormones etc), I also said LTP
What is meant bu Boolean algebra?
Data with only two possible values (think of a neuron that can only produce output which is either 1 or 0) is handled by a branch of mathematics called Boolean algebra, where the values 1 and 0 are often replaced by the labels true and false.
Boolean algebra comes with a set of basic operations or functions, which take as input boolean values and produce boolean output. Give three examples of these
AND, OR, XOR:
OR function, which given two boolean inputs, returns True if at least one of the inputs is True, or the exclusive or (XOR) function, which returns True only if exactly one of the two inputs is true. AND returns true only if both are true
For what inputs of x and y would activate these three functions?
Can be visualised with a truth table:
X | Y | X and Y | X or Y | X xor Y
0 | 0 | 0 | 0 | 0
1 | 0 | 0 | 1 | 1
0 | 1 | 0 | 1 | 1
1 | 1 | 1 | 1 | 0
How are these 1s and 0s represented in Python and other coding languages?
True
Most programming languages implement a boolean data type, which can be represented by 0 and 1 or by True and False labels. In Python, True and False are built-in constants, and we can ask Python to combine them with Boolean operations:
a = True
b = False
print(a or b)
We can also use the numeric values 1 and 0, as long as we tell Python that we mean boolean and not integer values.
a = bool(1)
b = bool(0)
print(a and b)
In electronics, what is a logic gate?
In electronics, a logic gate is a (conceptual) device which implements a boolean function. We can think of neurons as biological logic gates, in the sense that we can consider our modelled neuron as a system which receives binary inputs (e.g. spike or no spike from its presynpatic neurons) and outputs a binary signal (spike or no spike).
How do we broadly programme a neuron at the beginning of assignment 2?
We begin by writing a very simply Python class which we call neuron. This neuron has two methods, namely __init__, which is used to generate instances of this class, and output, which produces the neuron’s output given the inputs.
