Simple Neural Networks (PowerPoint)

Question 1

What is a Single Layer Perceptron (SLP)?

Answer 1

The simplest form of a neural network.

It consists of only one layer of artificial neurons, with each neuron connected directly to the input nodes.

Question 2

What sort of problems can a Single Layer Perceptron (SLP) solve?

Answer 2

Classification with one boundary/separation between groups (linear patterns, meaning that classes can be separated by a straight line).

Ex. Does the patient have a disease or not?

Question 3

What are the pros and cons of Single Layer Perceptrons (SLPs)?

Answer 3

Pros:
- Simple and easy to implement
- Training time is relatively fast compared to other neural network architectures
- Good for binary classification tasks and linearly separable problems
- Can be used as a building block for more complex neural network models

Cons:
- Limited to linearly separable problems and cannot learn complex decision boundaries
- Can only model a single output variable
- Not suitable for tasks that require multi-class classification
- Can suffer from the vanishing gradient problem when training on deep networks (the early layers don’t get enough feedback to adjust their weightd)
- Prone to overfitting (memorize the data set instead of understand the broader concepts) on noisy data or data with high dimensionality (data with a lot of different features or characteristics)

Question 4

What in the perceptron represents the synapses?

Answer 4

The weights

Question 5

What are two ways one can update the weights in a SLP?

Answer 5

1. Manually select the weight values: Compare network output y (ANN prediction) to expected output t (ANN target)
2. Use a learning/training rule

Question 6

What is the update rule for SLP?

Answer 6

The Update Rule: 𝑤’ = 𝑤 + 𝑎𝑥(𝑡-𝑦)
- 𝑤’: Updated weight after applying the update rule.
- 𝑤: Current weight.
- 𝑎: Learning rate, a constant that determines the step size during weight updates. (how quickly/slowly the model learns)
- 𝑥: Input to the neuron.
- 𝑡: Target (true) output.
- 𝑦: Actual output produced by the neuron

Question 7

What is a Multi-Layer Perceptron (MLP)?

Answer 7

An artificial neural network (ANN) that consists of multiple layers of nodes, including
- Input Layer
- One or more Hidden Layers
- Output Layer

Question 8

What is the activation function for a SLP? What is its biological counterpart?

Answer 8

The activation function is typically: y = f(x1 * w1 + x2 * w2)

It represent the soma (cell body).

Question 9

What is the biological counterpart for the inputs in a SLP?

Answer 9

The inputs, typically x1 and x2, represent neurons.

Question 10

What is the key problem when using SLP for classification?

Answer 10

Key problem is to learn the parameters that finds the boundary that sucessfully classifies the data items.