Neural Network Flashcards
What is a Neural Network?
A set of connected INPUT/OUTPUT UNITS, where each connection has a WEIGHT associated with it.
What is the fundamental processing element of a neural network?
A neuron.
What are the four main functions a neuron performs?
- Receives inputs from other sources
- Combines them in some way
- Performs a generally nonlinear operation on the result
- Outputs the final result
When did the backpropagation algorithm appear, marking a resurgence in neural networks?
1986.
What is meant by ‘Connectionist learning’?
Neural Network learning, due to the connections between units.
What are the characteristics of neural networks?
- Inputs are flexible
- Target function may be discrete-valued, real-valued, or vectors of values
- Outputs are real numbers between 0 and 1
- Resistant to errors in training data
- Long training time
- Fast evaluation
- Difficult for humans to interpret
What are the two basic normalization techniques for neural networks?
- Max-Min normalization
- Decimal Scaling normalization
What does Max-Min normalization do?
Maps a value to a specified range.
How is Decimal Scaling normalization achieved?
By moving the decimal point of values.
What is the purpose of bias in a neuron?
To transform the weighted sum from the origin.
What is the formula for the weighted sum in a neuron?
v = ∑wixi + b.
What is a Perceptron?
The basic unit in a neural network that acts as a linear separator.
What is the structure of an Artificial Neural Network (ANN)?
Interconnected nodes (artificial neurons) organized into layers.
What are the two main types of neural network connectivity?
- Feedforward networks
- Recurrent networks
What is the difference between single-layer and multi-layer feed-forward networks?
Single-layer has only an input layer projecting into the output layer, while multi-layer includes one or more hidden layers.
What is a recurrent network?
A network with feedback, where some inputs are connected to some outputs.
Fill in the blank: A neural network can learn to classify data by adjusting the _______.
weights.
True or False: Neural networks have a low tolerance to noisy and incomplete data.
False.
What is a key limitation of single-layer networks?
They cannot model complex relationships as effectively as multi-layer networks.
What is the primary goal of normalization in neural networks?
To ensure all values of attributes are within a specific range.
What is the relationship between the number of neurons in a human brain and an ant brain?
A human brain has 100 billion neurons, while an ant brain has 250,000 neurons.
What type of learning does a neural network primarily utilize?
Supervised learning.
What is the basic information processing unit of a neural network?
Neuron
A neuron consists of links with weights, an adder function, and an activation function.
What are the components of a neuron?
- A set of links (inputs with weights)
- An adder function (linear combiner)
- An activation function
What is the purpose of the activation function in a neuron?
To limit the amplitude of the neuron output
What distinguishes a multilayer feed-forward neural network?
It contains input, hidden, and output nodes, with no weights cycling back to previous layers.
What is a neurode?
Units in the hidden layers and output layer of a neural network
How many layers does a three-layer neural network contain?
Two hidden layers
What is the input layer of a multilayered feed-forward network composed of?
As many nodes as non-class attributes
What does the output layer correspond to in a neural network?
The class attribute, with as many nodes as classes
What is the process of learning in a neural network?
Inputs are fed into the input layer, processed through hidden layers, and produce outputs in the output layer.
What is backpropagation in neural networks?
A learning algorithm that iteratively processes training data to minimize classification error.
What is the first step in the backpropagation algorithm?
Initialize the weights and biases to random numbers from the interval [-1,1].
What is the purpose of the bias in a neural network unit?
To adjust the output along with the weighted sum of inputs.
What happens during the forward propagation step?
Compute the net input and output of each unit in the hidden and output layers.
Fill in the blank: The error is propagated backwards by updating weights and biases to reflect the _______.
error of the network classification
What is case updating in neural networks?
Updating weights and biases after each sample presentation.
What is an epoch in neural network training?
One iteration through the training set.
What are the terminating conditions for training a neural network?
- Errors below a threshold
- Percentage of misclassified samples below a threshold
- A pre-specified number of epochs has expired
What does network pruning refer to?
Removing some links in a network without affecting classification accuracy.
What is a modular neural network?
A combination of several neural networks to reduce load.
What is supervised learning in the context of neural networks?
A type of learning where the model is trained on labeled data.
What is the difference between constructive and destructive algorithms in neural network training?
- Constructive: Build up new layers and nodes during training
- Destructive: Prune unnecessary layers and nodes during training
What is the primary goal of the training process in a multilayer perceptron (MLP)?
To minimize the root mean square (RMS) error.
What is one method to achieve faster convergence in neural networks?
Stochastic learning, updating weights after each training example.
List some applications of neural networks.
- Handwritten digit recognition
- Face recognition
- Time series prediction
- Optical character recognition
What are some applications of neural networks in medicine?
- Analysis of electrocardiogram data
- RNA & DNA sequencing
- Drug development without animal testing
What is a basic application of Neural Networks in image processing?
Handwritten Digit Recognition
This application involves training neural networks to identify and classify handwritten digits.
What is one application of Neural Networks in finance?
Forecasting/Market Prediction
Neural networks can analyze historical data to predict market trends.
What is a common application of Neural Networks in manufacturing?
Quality control, fault diagnosis
Neural networks can help in identifying defects in products and diagnosing faults in machinery.
In medicine, how are Neural Networks applied?
Analysis of electrocardiogram data, RNA & DNA sequencing, drug development without animal testing
These applications use neural networks for analyzing medical data and aiding in research.
What are the main components discussed in the summary of Neural Networks?
Basic building block of Artificial Neural Network, Construction, working and limitation of single layer neural network, Back propagation algorithm, Advanced Features, Application of Neural Network
These components are essential for understanding the structure and function of neural networks.
What is a characteristic of neural networks regarding hidden units?
More hidden units generally produce lower error
Increasing the number of hidden units can enhance the model’s ability to learn complex patterns.
What is a challenge in determining network topology?
Determining network topology is difficult
The optimal structure of a neural network is often not straightforward and requires experimentation.
Why is choosing a single learning rate for neural networks considered impossible?
Choosing single learning rate impossible
Different parts of the network may require different learning rates for optimal performance.
What is a limitation in reducing training time for neural networks?
Difficult to reduce training time by altering the network topology or learning parameters
Adjusting these factors does not guarantee a significant reduction in training duration.
What is a notable feature of neural networks that often leads to better results?
NN(Subset) often produce better results
Using subsets of data for training can enhance the performance of the neural network.
What algorithm is used for training multi-layer feed forward neural networks?
Back propagation algorithm
This algorithm helps in updating the weights of the network to minimize errors.
What is the limitation of a single layer neural network?
Limited capacity to model complex functions
Single layer networks can only solve linearly separable problems.
Fill in the blank: The basic building block of an Artificial Neural Network is _______.
[Neuron]
Neurons are the fundamental components that process inputs and produce outputs in a neural network.
