Connectionism

Question 1

What is connectionism?

Answer 1

A paradigm in cognitive science that states that human mental processes can be explained by the computational / artificial modelling of neural networks.

Question 2

What is the starting point of connectionism?

Answer 2

The architecture of the brain, neurons, neural networks.

Question 3

What are neural networks?

Answer 3

• Interconnected neurons distributed across the brain.
• Parallel and distributed (whereas the Turing machine is serial and modular).
• Examples: Face recognition, a rolling red ball.

Question 4

What is an artificial neural network?

Answer 4

A computational model of a simplified version of a biological neural network. Simulates networks of interconnected neurons in the brain. Learns to represent the world in this network structure and deliver outputs.

Question 5

What are connectionist models also called?

Answer 5

Parallel distributed processing (PDP) models.

Question 6

What are the three representations in connectionism?

Answer 6

• Sub-symbolic (rather than symbolic, i.e. no explicit symbolic representation of rules and properties, nodes representing neurons)
• Distributed / parallel (rather than local/modular)
• Superposition (rather than discrete/categorical, i.e. the system is able to be in multiple states at the same time until the output is provided)

Question 7

What are the characteristics of connectionist models?

Answer 7

Characterized by connections and different strengths of connection between processing units.

Processing units are meant to represent neurons and communicate with one another by signals (such as firing rate).

Three layers: input units, hidden units and output units.

Question 7

What is the goal of connectionist models?

Answer 7

To simulate the actions of interconnected neurons in the brain. To perform a task by specifying the architecture, i.e. the number of units, their arrangement in layers and columns, the pattern of connectivity and the weight/strength of each connection.

Question 8

What are the input units?

Answer 8

Input units receive stimuli from the environment and get activated.

Similar to how the dendrites of the neuron receive input from other neurons.

Some biological factors are distance to the soma and concentration of neurotransmitter receptors.

The input is summed at the axon hillock and the dendrite activity is graded. Either the axon fires or it does not, i.e. “all-or-none”.

Question 9

What are the hidden units?

Answer 9

The hidden units receive signals from the input units. The hidden units provide a state space for compressing the data.

Question 10

What are the output units?

Answer 10

The output units receive signals from the hidden units.

The output can be recognition of an object, a word etc.

Question 11

What is a connection weight?

Answer 11

A connection weight determines how signals sent from one unit either increase or decrease the activity of the next unit (just like exhibitory or inhibitory synapses).

Question 12

What does back propagation do?

Answer 12

Backpropagation sends error signals back through the system and adjusts the weights.

Question 13

Mention the components of a connectionist model.

Answer 13

Input units, hidden units, output units, connection weights, backpropagation.

Question 14

What are the 4 types of activation functions in a connectionist model?

Answer 14

Identity, step, threshold logic and sigmoid.

Question 15

What is a state space?

Answer 15

The possible states that a complex system such as the brain or a neural network can be in.

E.g. The game of chess, i.e. the state space is all the states the game could be in.

Question 16

What is a vector space?

Answer 16

Any point within the state space defines the activation of the nodes in the hidden layer.

Tells you everything about the state of hidden layer.

Question 17

What is trajectory?

Answer 17

Representation of how the system changes over time.

E.g. going from state 1, 2, 3 in the game of chess.

Change in activation vector over time.

Question 18

What are attractors?

Answer 18

A region of state space the system is confined to by all the constraints operating in the system.

Constrains the state space accessible to the system by adjusting the weights on the hidden units.

Question 19

What are the steps in a connectionist model?

Answer 19

1. Define the network, i.e. inputs, layers, nodes per layer, outputs. Apply random weights to hidden units to begin with.
2. Run model on data and record answer.
3. Compare the actual output to desired output. Is the output correct? In the first trials before the weights are changed, the output is most likely to be incorrect.
4. Adjust weights to minimize difference by the process of back propagation.
5. Repeat steps with new input.
6. Keep repeating the process and adjusting the weights until the outputs are consistently correct.

Question 20

Synaptic plasticity and their relevans for connectionist models?

Answer 20

In the connectionist model, the weights between units become stronger when the same connection is activated again and again. Memory is embodied in the unit weights.

The same thing happens in the brain. This is known as synaptic plasticity, i.e. the ability of synapses to change their strength.

The process is called long-term potentiation. When the same neural connections are activated multiple times, the connections are strengthened because the synapses between neurons are strengthened.

Question 21

What is the role of training and tests in connectionist models?

Answer 21

Connectionist models include both training and tests to figure out how accurate the model is. When the model is trained, the weights are either weakened or strengthen by back propagation. The models take a long time to learn, i.e. it takes many trials to be able to produce correct outputs.

Question 22

What are the strengths of connectionism?

Answer 22

1. Biologically more realistic image of the mind: Anatomically based, tries to replicate neurons, networks, synapses. If a unit breaks down, the system still works.
2. Relation between syntax and semantics: Semantics are represented syntactic.
3. Enables solving complex tasks, e.g. object recognition, planning, coordinated movement.
4. Allows fuzzy category boundaries/rules: e.g. albino tigers are still tigers even though they cannot be described as large black/orange animals.

Question 23

What are the weaknesses of connectionism?

Answer 23

1. Simplistic model of the brain: What about neurochemistry, different types of neurons?
2. Back propagation is anatomically unsupported: Neural signals are unidirectional.
3. Artificial neural network learning is slow and expensive.
4. Sometimes rule-based processing is more adequate for the task: e.g. reasoning and language.

Question 24

The example of a mine detector.

Answer 24

A neural network is trained so that it can detect mines from reefs.

The network is not able to distinguish mines from reefs to begin with.

It needs to be presented with a massive set of different sonar signals from mines and reefs. It requires lots of training.