VI: Artificial Neural Networks Flashcards
What are artificial neural networks?
Artificial neural network (ANN) is a set of connected neurons formed as a network. Neural networks apply algorithms in order to train models for data sets just as in machine learning, but with the addition that neural networks use several connected computational processes referred to as neurons. They are inspired by the structure of the brain. Artificial neural network is a group of artificial neurons connected in a topology joining several different neurons. They can be viewed as implementing complex machine learning algorithms.
What is the history of ANN?
Neural networks are models of how systems operate. The basic units are neurons and the most common neural model is the same as the first one from 1941, which is a nice tribute to the work that preceded the field. McCulloch and Pitts created ANN by creating a computational model called threshold logic and defined an artificial neuron that is still commonly used by many neural net systems. This neuron could understand inputs and outputs.
What do the simplest ANN networks consist of? How do more complex networks look?
They consist of a single artificial neuron with a set of inputs and a single output. A more complex multi-layered network, where the neurons are grouped into layers that can be fully connected.
What are perceptrons and support vector machines?
They are single-layer ANN networks.
When are ANN useful?
In instances where there is a high complexity of the data or tasks makes it very hard to create it by impractical. They can perform interior functions from observations, and can handle data processing like filtering and clustering.
What are classical ANN applicable in?
They are applicable in fields where data sets are multi-dimensional but can be trained on a few numbers of layers:
Predicting the stock market
Loan applications
Predicting winning teams
Image recognition
What is a McCulloch-Pitts neuron?
McCulloch-Pitt’s neuron has a set of input, one is biased, and an output. The bias can be viewed as a constant term that is generally added and meaning a threshold used to determine if a neuron will become activated. The reason why bias is applied is to carry out mathematical computation for training the network and creating a model. Bias helps the network train and makes it easier for neurons to fire and not fire for the same input values.
What is an activation function? What different types are there?
An activation function is a mathematical function that is applied to the value of the activation level, making neurons active. The activation function maps the sum of the activation level to a function equation, providing an output of the neuron:
- Sigmoid is the most common activation function, easy to analyze and easy to calculate.
- Hyperbolic tangent
- Heaviside step
Why apply bias?
Bias is applied to carry out the mathematical computation for training the network and creating a model over the data set. It cannot be 0, since a random weight is multiplied to the bias, which is why 0 does not make any sense.
What is a sigmoid activation function?
Sigmoid is the most common activation function, easy to analyze and easy to calculate. It maps the weighted sum of inputs to a value between 0 and 1, allowing the output to represent a probability or a binary decision. The function has an S-shaped curve, gradually transitioning from 0 to 1 as the input increases. It is defined as f(x) = 1 / (1 + exp(-x)). The sigmoid activation function is useful for binary classification problems, but it suffers from vanishing gradients and tends to saturate for extreme input values, which can hinder learning in deep neural networks.
What is a hyperbolic tangent?
The hyperbolic tangent (tanh) activation function is a non-linear activation function commonly used in neural networks. It maps the weighted sum of inputs to a value between -1 and 1, providing a smooth transition between negative and positive values. The tanh function is similar to the sigmoid function but centered at 0, making it suitable for tasks where the output range needs to be symmetric around zero. It retains the non-linear properties of the sigmoid function but avoids some of its limitations. However, like the sigmoid function, the tanh function can also suffer from vanishing gradients for extreme input values.
What is a single-layer neural network?
Single-layer neural networks, also known as perceptrons, consist of only one layer of artificial neurons. They are the simplest form of neural networks and can be used for binary classification tasks. Each neuron in the network receives input signals, applies weights to them, and computes a weighted sum. Then, an activation function is applied to the sum to produce the output of the neuron. The weights and biases in the network are adjusted during training to optimize the model’s performance. However, single-layer neural networks are limited in their ability to handle complex patterns and non-linear relationships.
What are multi-layer neural networks?
Multi-layer neural networks, also known as deep neural networks, consist of multiple layers of artificial neurons. These networks are designed to handle complex patterns and learn intricate representations. The input layer receives input signals, which are then passed through hidden layers that perform computations using weighted connections. Each hidden layer applies an activation function to produce output. The final layer, known as the output layer, generates the final predictions or outputs. Deep neural networks employ backpropagation to adjust the weights and biases during training, optimizing the network’s ability to learn and make accurate predictions across various tasks.
What is supervised learning and processing?
Supervised learning backpropagation follows the principle of gradient descent. The weights are modified towards a negative gradient of an error measure, minimizing the gap between the calculated value and the desired value. Supervised learning is a machine learning approach where an algorithm learns from labeled training data. The algorithm is trained to predict or classify new, unseen instances based on the patterns observed in the labeled examples. The process involves mapping input data to corresponding output labels, enabling the algorithm to make accurate predictions on unseen data. During training, the algorithm adjusts its internal parameters to minimize the difference between predicted and actual outputs. Supervised learning is commonly used in various tasks such as regression (predicting continuous values) and classification (predicting class labels).
What is backpropagation for training neural networks?
Backpropagation is a popular training algorithm for neural networks. It involves a two-phase process: forward propagation and backward propagation. In forward propagation, input data is fed through the network, and the outputs are computed. Then, the error between predicted and actual outputs is calculated. In backward propagation, the error is propagated back through the network, adjusting the weights and biases of each neuron based on their contribution to the error. This iterative process continues until the network learns to minimize the error and make accurate predictions. Backpropagation allows neural networks to learn and improve their performance through adjusting their internal parameters.
