M2 Flashcards
Artificial Neural Networks (ANN) is also known as (3)
Neural networks
Neural computing (or neuro-computing) systems
Connectionist models
___ simulate the biological brain for problem solving
ANNs
The biological brain is a massively ___ system of interconnected processing elements
parallel
ANNs simulate a similar network of simple processing elements at a greatly ___ scale
reduced
ANN is first developed in the 19__ and ___
1950s and 1960s
There is a great upsurge in interest in Artificial Neural Networks since the mid ___s
1980s
Both ANNs and expert systems are ___ tools for problem solving
non-algorithmic
ES rely on the solution being expressed as a set of heuristics by an expert, ANNs learn solely from ___
data
There is estimated ___ neurons in the human brain, with each connected to up to 10 thousand others
1000 billion
Electrical impulses produced by a neuron travel along the ___
axon
The axon connects to ___ through synaptic junctions
dendrites
A neuron adds its inputs and “fires” (produces an output) when the sum of its inputs exceeds a certain threshold
sum
The strengths of a neuron’s inputs are modified (enhanced or inhibited) by the ___
synaptic junctions
Learning in our brains occurs through a continuous process of new interconnections forming between neurons, and ___ at the synaptic junctions
adjustments
A simple model of the ___, first proposed in 1943 by McCulloch and Pitts
biological neuron
A Synthetic Neuron consists of a summing function with an internal threshold, and “___” inputs
weighted
In Synthetic Nueron, for a neuron receiving n inputs, each input xi ( i ranging from 1 to n) is weighted by multiplying it with a ___
weight wi
In Synthetic Neuron, the sum of the wi xi products gives the ___ of the neuron
net activation
In Synthetic Neuron, the activation value is subjected to a ___ to produce the neuron’s output.
transfer function
In Synthetic Neuron, the weight value of the connection or link carrying signals from a neuron i to a neuron j is termed ___
wij
___ compute the output of a node from its net activation
transfer function
Step function
Signum function
Sigmoid function
Hyperbolic tangent function
these are the popular ___
transfer functions
In the ___, the neuron produces an output only when its net activation reaches a minimum value – known as the threshold
step function
When the threshold T is 0, the step function is called ___.
signum
The ___transfer function produces a continuous value in the range 0 to 1
sigmoid
___ is s variant of the sigmoid transfer function wherein its shape is similar to the sigmoid (like an S), with the difference that the value of outputi ranges between –1 and 1
Hyperbolic Tangent
The building block of an ANN is the ___
artificial neuron
The most common architecture of an ANN consists of two or more layers of artificial neurons or nodes, with each node in a layer connected to every ___ in the following layer
node
In ANN, signal usually flows from the ___ layer, which is directly subjected to an input pattern, across one or more hidden layers towards the output layer
input
The most popular ANN architecture is known as the ___
multilayer perceptron
In some models of the ANN, such as the ___ or Kohonen net, nodes in the same layer may have interconnections among them
self-organising map (SOM)
The input stimulus of an ANN are data values grouped together to form a ___
pattern
The output value(s) of nodes in the output layer represent the ___ of the input pattern
category
Any ___-layer ANN can (at least in theory) represent the functional relationship between an input pattern and its class
three
The process by which an ANN arrives at the values of these weights is known as ___
learning or training
Learning in ANNs takes place through an iterative training process during which node interconnection ___ values are adjusted
weight
___weights, usually small random values, are assigned to the interconnections between the ANN nodes
Initial
____ in ANNs can be the most time consuming phase in its development
learning
In supervised learning of ANN, the weight adjustments during each iteration aim to reduce the “___” (difference between the ANN’s actual output and the expected correct output)
error
In ANN supervised training, pairs of sample input value and corresponding output value are used to train the ___ repeatedly until the output becomes satisfactorily accurate
net
In ANN unsupervised training, the net adapts itself to align its weight values with training patterns. This results in groups of nodes responding strongly to specific ___ of similar inputs patterns
groups
A neural network can be in one of two states: (2)
training mode or operation mode
Most ANNs learn ___ and do not change their weights once training is finished and they are in operation
off-line
In an ANN capable of ___ learning, training and operation continue together
on-line
What are the three most well known models of ANN
- The multilayer perceptron
- The Kohonen network (the self-organising map)
- The Hopfield net
Type of ANN where nodes are arranged into an input layer, an output layer and one or more hidden layers
Multilayer Perceptron
The multilayer perceptron is also known as the ___ network because of the use of error values from the output layer in the layers before it to calculate weight adjustments during training
backpropagation
Another name for Multilayer Perceptron is ___
Feedforward network
The learning rule for the multilayer perceptron is known as “___” or the “backpropagation rule”
the generalized delta rule
The generalized delta rule repetitively calculates an error value for each input, which is a function of the ___ difference between the expected correct output and the actual output
squared
___ = Old weight + change change calculated from square of error
New Weight
__ is the difference between desired output and actual output
Error
Training stops when error becomes acceptable, or after predetermined number of ___
iterations
In MLP. for a given pattern p, the error Ep can be plotted against the weights to give the so called ___
error surface
The error surface is a landscape of hills and valleys, with points of minimum error corresponding to ___ and maximum error found on ___
wells, peaks
MLP follows the method of ___ where the changes are made in the steepest downward direction
gradient descent
In MLP, all possible solutions are depressions in the error surface, known as ___
basins of attraction
The MLP may fail to settle into the global minimum of the error surface and instead find itself in one of the ___
local minima
A number of alternative approaches can be taken to perturb the ANN out of local minima in MLP such as, Lowering the gain term progressively, Addition of more nodes for better representation of patterns, Addition of random noise to perturb the ANN out of local minima, and Introduction of a ___
momentum term
Lowering the ___ progressively is used to influence rate at which weight changes are made during training. The value by default is 1, but may be gradually reduced to reduce the rate of change as training progresses
gain term
Introduction of a ____ determines effect of past weight changes on current direction of movement in weight space and is also a small numerical value in the range 0
momentum term
Addition of random ___ to perturb the ANN out of local minima is usually done by adding small random values to weightso and takes the net to a different point in the error space
noise
is a biological systems that display both supervised and unsupervised learning behavior
The Kohonen Network (the self-organizing map)
During training, the Kohonen net changes its weights to learn appropriate ___, without any right answers being provided
associations
The Kohonen net consists of an input layer, which distributes the inputs to each node in a second layer, known as the ___
competitive layer
In Kohonen Network, neurons in the competitive layer have ___ (positively weighted) connections to immediate neighbors and ___ (negatively weighted) connections to more distant neurons
excitatory, inhibitory
As an input pattern is presented, some of the neurons in the competitive layer are sufficiently activated to produce outputs, which are fed back to other neurons in their ___
neighborhoods
In Kohonen Network, the node with the set of input weights closest to the input pattern component values produces the largest output. This node is termed the ___
winning node
The ___ is the most widely known of all the auto-associative ANNs
Hopfield net
In ___, a noisy or partially incomplete input pattern causes the network to stabilize to a state corresponding to the original pattern
auto-association
The Hopfield model is also useful for ___tasks
optimization
The Hopfield net is a ___ ANN in which the output produced by each neuron is fed back as input to all other neurons
recursive
The Hopfield net has no iterative learning algorithm as such. ___ (or facts) are simply stored by setting weights to lower the network energy
Patterns
During operation of Hopfield Net, an input pattern is applied to all neurons ___and the network is left to stabilize
simultaneously
Parallelism
