305 Flashcards
What does the Artificial Neural Networks constitute
A part of computer science based on neuroscience ideas
What is biological excitation
Internal mechanism
What are the external neurons that enable the propagation
External mechanism
What are propagated by a neuron
Spikes
Describe a neuron
Able to propagate signals over large distances
Propagate information by generating electrical pulses (action potentials or spikes) that can travel down nerve fibres.
Specialised for generating electrical signals in response to chemical/other inputs and transmitting them to other cells
Represent and transmit information by firing a sequence of spikes in various temporal patterns
What is the electrical potential between in most living cells
Interiors and (exterior) environment
What is one of the factors determining the energy barriers encountered by charged substances (ions) entering/leaving the cell
The membrane potential
What are ion channels
Proteins within the cell membrane with the central pore through which ions can cross the membrane
What is the function of the cell membrane
It acts as a barrier for ions
What elements are ions predominantly made of
Sodium
Potassium
Calcium
Chloride
How do Ion channels control the flow of ions
By opening and closing in response to voltage changes and both internal/external signals
What causes the response of a large amplitude electrical wave
A significantly large perturbation, above a threshold in intensity and duration
Describe wave propagation in biological excitation
Travels with uniform velocity
Excitation/transmission is all or none- strength does not vary
Excitation is followed by the absolute refractory period, an unexcitable period of definite duration
What is the function of dendrites
Receive inputs from many other neurons through synaptic connections
What is the soma
The cell body
What is the function of the axon
Carries signals from the neuron to other neurons/effectors
What is the name for the tips of axon branches
Boutons/nerve terminals
What is a synapse
The location of interaction between a terminal and the cell
When is a neurotransmitter released
When a spike arrives from the presynaptic neuron
Describe trans-synaptic stimulation
Neurotransmitters cross the synaptic cleft and bind to receptors on the dendrite spine
Excitatory synapses on the cortical pyramid form on dendrite spines or axon
Describe no physiological continuity from neuron to neuron
When an impulse (perturbation) reaches a
synapse, it does not necessarily stimulate the
following neuron
What does trans-synaptic stimulation of a neuron require
Either:
Temporal summation- a repetition of impulses in time at the same synapse
or
Spatial summation- the simultaneous arrival of impulses at a sufficient number of adjacent synapses to make the density of excitation high enough
What is inhibition
The opposite effect of excitation, renders the element less excitable to other stimuli. Can occur due to the arrival of an impulse at synapses
What is the top trace
A recording from an intercellular electrode connected to the soma of the neurone.
What is the middle trace
A simulated extracellular recording. Action potentials appear roughly equal positive and negative potential fluctuations with an amplitude of 0.1V, 1000x smaller than intracellularly recorded action potential
What is the bottom trace
A recording from an intracellular electrode connected to the axon some distance away from the soma.
Describe neuron to neuron propagation
Spikes, but not subthreshold potentials, propagate regeneratively down the axons
What is the weight of connection
The connection strength, assigned some value w, that describes the importance of a connection
What represents the dendrites in the abstract neuron model
Inputs a(0) to a(n)
What represents the soma in the abstract neuron model
The neuron body, represented by a circle
What represents the axon in the abstract neuron model
Output X
What represents the excitation potential threshold in the abstract neuron model
The threshold
What represents the importance of presynaptic neurons in the abstract neuron model
Weights of input connections
When is the abstract neuron excited
When the weighted sum is above the threshold 0
What did McColloch and Pitts demonstrate
Because of the all or none character of nervous activity, neural events and the relations among them can be treated by means of the propositional logic
How was the neuron modelled by McColloch and Pitts
As a discrete time input
With excitatory and inhibitory connections and an excitation threshold
Describe discretization
Comparable to the refractory period
No zeno executions
Describe fixed time step size
The impulse travels with a nearly uniform velocity in a biological neural system
What is the value of w for an excitatory connection
+1
What is the value of w for an inhibitory connection
-1
What is the function of a register cell
To retain the input for one period elapsing between 2 instances
What is the excitation threshold in a McCulloch-Pitts neuron equivalent to in a biological neuron
Potential threshold
What prevents excitation of an MP neuron
Activity of a single inhibitory input (input via a connection with negative weight)
What is S ^(t-1)
Instant state of the neuron
What is each stage of the MP Neuron Computation Algorithm
Check inputs from all inhibitory connections
If they are not all 0, Xt = 0, else:
Calculate the instant state of the neuron
If S^(t-1) < threshold Xt = 0
else (S^(t-1) >= threshold): Xt = 1
What is linear separability
There exists a line/plane such that all inputs which produce a 1 for the function lie on one side of the line/plane and all inputs which produce a 0 are lie on the other side of the line/plane
Can each linearly separable function be represented by a single MP neuron (Complete)
No
Define learning
To change in response to experience
What is the ANN learning rule
The rule how to adjust the weights of connections to get desirable output
What was Hebb’s thesis
Cells that fire together, wire together
What is the simple formulation of Hebb’s rule
Increase weight of connection at every next instant in the way
When is there an excitatory input according to Hebb’s rule
When input is not equal to 0
When is the neuron fired according to Hebb’s rule
When output is not equal to 0
Describe how to normalise inputs for Oja’s rule
Square route the sum of all the inputs
-root(w1 + w2 +… Wn)
Set weights to 1/result multiplied by the previous weight
Continue until max change in weight <= convergence criteria
Describe unsupervised learning
A type of machine learning where the algorithm is not provided with any pre-assigned labels or scores for the training data.
As a result any unsupervised learning algorithms must first self-discover any naturally occurring patterns in the training data set
Describe clustering
An unsupervised network that can group similar sets of input patterns into clusters predicated on a predetermined set of criteria relating to the components of the data
When can clustering be achieved
When we extend the single neuron to the network with multiple outputs
Describe competitive learning
We consider a one layer neural network with multiple outputs
A single output neuron of a network competes among all the outputs to have its weight updated, whereas in Hebbian learning several outputs can be simultaneously updated
What is competitive learning suited for
Discovering statistically important features that may be used to classify sets of input patterns
Describe what a Self Organising Map is used for
Used to produce a low dimensional (typically 2D) representation of a higher dimensional data set, while preserving the topological structure of the data.
Can be used for clustering or visualisation
Define competitive learning
An output neuron competes among all the outputs to be updated
Only a single output neuron is updated in an instance
What is competitive learning suited for
Discovering statistically important features that may be used to classify a set of input patterns
How do you calculate delta w for the Kohonen rule
C( a(i) - w(ji) )
Which output neuron is updated according to the Kohonen rule
The output neuron with the maximum value Sj at that instant
The winner
How do you calculate the incremental term (delta w) for a self organising map
C( a(i) - w(ji) ) x theta(j, j)
where theta(j, j) is the restraint function due to the distance between j and j*
Describe the process of neuron excitation in a self organising map
In the map, location of the most strongly excited neurons (winner) is correlated with the certain input signals
Neighbouring excited neurons correspond to inputs with similar features
Because in the training phase the whole neighbourhood of neurons are moved in the right direction, similar items tend to excite adjacent neurons. Therefore, SOM forms a semantic map where similar samples are mapped close together and dissimilar ones apart
What is the difference between an MP neuron and a Hebbian neuron
Learning rule: The weights are updated in the Hebbian neuron
Output neuron: MP uses threshold value, Hebbian uses continuous activation function
A Hebbian neuron can learn from the input data
Hebbian neuron weights can be arbitrary numbers, MP weights are 1 or -1
What kind of learning is Hebb’s rule
Unsupervised
-unlabelled inputs
-does not rely on knowing correct output for learning process
What kind of learning is Kohonen’s rule
Unsupervised
-input is labelled
Competitive
-Only a single output is updated at any instant
Describe the initialisation step of the learning process for a self organising map
Assign random weights to each neuron
Set learning rate C
Define restraint function to adjust weights
What are the 2 criteria for competition for a self organising map
Either:
Calculate the Euclidian distance between the input vector and the weights of each neuron
Winner is neuron with smallest distance
Winner is the neuron with the highest weighted sum
What are the 4 steps of the SOM learning process
Initialisation
Competition
Weight update
Iteration
