WK 8 - Neural Networks 1 (AI, The Brain and Neural Computing)

Question 1

Brain Structure

Answer 1

• Contains around 100 billion neurons
• Neurons communicate through synapses – effectively a configurable chemical junction between neurons

Question 2

Neurons parts

Answer 2

Dendritic tree: receive signals

Cell body: process signals

Axon: transmit signals

Question 3

Neuronal Function

Answer 3

A neuron receives electrical activity from other neurons along its dendrites (inputs)

The axon (effectively the output of the neuron) will produce a pulse based on the strength of the incoming pulse

This is then passed to other neurons connected to this one

Question 4

Synapse

Answer 4

chemical junction which can be modified and therefore is thought to be where learning takes place

The synapse can release more neurotransmitter to enhance the coupling between cells

Question 5

Artificial Neuron

Answer 5

When two neurons fire together the connection between the neurons is strengthened
The activity of firing is one of the fundamental operations necessary for learning and memory

Question 6

Rosenblatt’s Perceptron Architecture

Answer 6

The Perceptron consists of a single layer of artificial neurons or “perceptrons.”

Each perceptron takes a set of input features and produces a binary output (0 or 1).

Question 7

Rosenblatt’s Perceptron Training Data:

Answer 7

The training data for the Perceptron algorithm consists of labeled examples, where each example is
epresented by a set of input features and a corresponding target class label (0 or 1).

Question 8

Rosenblatt’s Perceptron how it works:

Answer 8

the system was able to learn by means of weighted
connections

Question 9

Problems with the Rosenblatt’s Perceptron

Answer 9

The Perceptron algorithm can only learn and classify linearly separable data.

Binary Classification : The Perceptron algorithm is designed for binary classification tasks, where it assigns instances to one of two classes.

Perceptron could not correctly solve the
XOR function.

Question 10

Connectionism

Answer 10

Add a further layer of neurons to the network and create a Multi-Layer Perceptron to resolve the XOR issue of teh single layer perceptron.

Question 11

Neural computing

Answer 11

Neural Computing is based on artificial neural networks (ANNs) that consist of interconnected nodes (neurons) and learn from data through training algorithms. ANNs are inspired by the structure and functioning of biological neural networks in the brain.

Question 12

Traditional AI:

Answer 12

Traditional AI encompasses various techniques such as symbolic logic, rule-based systems, expert systems, and search algorithms. It focuses on explicit
epresentation of knowledge and logical reasoning.

Question 13

Artificial Neural Networks (ANNs) learning :

Answer 13

• Supervised Learning
• Unsupervised Learning

Question 14

Supervised Learning

Answer 14

machine learning approach where:
- artificial neural networks are trained using labeled input-output
-The network then corrects itself based on that output adjusting its internal parameters (weights and biases) during training to minimize the difference between predicted outputs and actual outputs

Question 15

Unsupervised Learning

Answer 15

-The network organises itself according to patterns in the data
-No external ‘desired output’ is provided

Question 16

Perceptron

Answer 16

Consists of a set of weighted connections, the neuron
(incorporating the activation function) and the output axon

Question 17

Modified Versions of Percepron Learning

Answer 17

The larning can be slowed down with a decimal term between 0 and 1 when the weight is updated.

Question 18

Widrow-Hoff Learning Rule

Answer 18

– weight updates proportionate to
the error made

Δ = desired output – actual output
w_i (t+1) = w_i (t) + ηΔx_i(t)

Question 19

Limitations of the Perceptron

Answer 19

• can only solve linearly separable
  problems ( cannot do it for XOR)

Question 20

Multi-Layer Perceptron

Answer 20

Single percepron limitation can be overcome by adding a further layer to the network.

Three layers
* Input
* Hidden
* Output

Question 21

Activation Functions

Answer 21

• Sigmoid function:
  The steepness of the curve is changed by z
  The derivative can be easily
  computed

Question 22

Weights

Answer 22

• are variable strength connections between
  units
• propagate signals from one unit to the next
• main component changed during learning

Question 23

FeedForward - supervised learning algorithm

Answer 23

type of neural network architecture where information flows in one direction, from the input layer to the output layer, without cycles or loops.

Question 24

Feed Forward output calculation

Answer 24

‒ Multiply incoming signal by weight
‒ Pass this through sigmoid activation function
‒ Pass on this output to units in the next layer

Question 25

Backpropagation ( updating rule )

Answer 25

Adapt the weights
Start from the output layer and work backwards

New weight (t+1) = old weight, plus a learning rateerror for pattern p on node joutput signal for p on j

Question 26

Two Types of Weight Updating

Answer 26

Batch Updating : All patterns are presented, errors are calculated, then the weights are updated

Online Updating : The weights are updated after the presentation of each pattern

Question 27

Momentum

Answer 27

• Momentum encourages the network to make large changes to weights if the weight changes are currently large
• This allows the network to avoid localminima in the early stages as it can overcome hills

weightupdatefunction + (w_ij (t) − w_ij (t −1))

Question 28

Symbolic AI

Answer 28

• represent knowledge in the form of symbols, rules, and relationships.
• manipulation and processing of symbolic representations of knowledge and logic.
• Expert System (IF-THEN rules)
• Enabling Reasoning
• Knowledge Programmed (by Humans)
• Serial (fragile)
• Does Not Generalise (outside scope)
• Understandable/Explainable

Question 29

Connectionism

Answer 29

Implicit Representation (numbers)
- Neural Network (weighted graph)
- Enabling Perception
- Knowledge Learned (from Data)
- Distributed (graceful degradation)
- Generalise (outside scope)
- Black-box

Question 30

Neural Network Properties

Answer 30

-Able to learn to relate input variables to required output
- Is able to generalise between samples
- Shows graceful degradation

Question 31

Classification

Answer 31

assigning input data to predefined categories or classes.

Question 32

Regression

Answer 32

involve predicting a continuous output value based on input variables.

Question 33

Graceful Degradation

Answer 33

symbolic systems: the removal of one component of the system results in failure

Removal of neuron(s) from a neural network: Reduce performance and probably not result in overall failure

Question 34

‘Generalisation’ in Symbolic AI

Answer 34

Extreamly difficult, Can operate as expert systems in constrained environments but will quickly fail if taken out of environment

Question 35

Generalisation in Neural Networks

Answer 35

Neural networks can learn common patterns in data therefore they can learn the distinctions between different classes of output

Question 36

Classification use

Answer 36

designed to group samples according to some known property.

– Minimum 2 datasets required – training and testing

Question 37

Data Representation Issues

Answer 37

• Continuous data: Good data type for neural networks
• Integer-type Data
  -Discrete Categories: Each value needs to have separate representation in the network

Question 38

categorical variables representations

Answer 38

• Field-type
• Thermometer-type
  -Missing Values

Question 39

Field-type:

Answer 39

Field-type representation is a method where each category is associated with a separate field or feature.

Each field represents a specific category, and the value of that field indicates the presence or absence of that category.

Question 40

Thermometer-type:

Answer 40

Thermometer-type representation is a method where each category is associated with a “thermometer” vector or bar.

Question 41

Thermometer-type vs Field

Answer 41

The difference between the two representations lies in the way categories are encoded.

The thermometer-type representation uses binary vectors to capture the degree of membership or association with each category,

while the field-type representation uses separate fields to indicate the presence or absence of each category.

Question 42

Missing Values

Answer 42

• Occur frequently in real world data
• Cannot be entered directly into the network
• Requires some value in each row

Question 43

Overfitting

Answer 43

Overfitting occurs when we train the network on a task for too long

The network learns the noise in the input as well as the
common patterns

The result is poor performance on unseen examples

Question 44

Early Stopping (to avoid overfitting)

Answer 44

• cross-validation

Have three sets, training, testing and cross-validation