Connectionism

Question 1

Neural network

Answer 1

Consists of an input layer, hidden layers and an output layer
Overall the network transforms any input pattern into a corresponding output pattern as dictated by the arrangement and strength of the many connections between neuro

Question 2

Chinese gym

Answer 2

Many Chinese room’s might solve the symbol grounding problem (but we need to many since 1 neuron = one person)

Question 3

Gradient descend

Answer 3

An optimization algorithm for finding a local minimum of a differentiable function
Used for back-propagation

Question 4

Solution to symbol grounding problem

Answer 4

Rises from complexity.
How we assign meaning lays beyond language and humans
One must look at how neurons code and transform sensory signals
the relations between weights and activations of the neurons of the trained model

Question 5

Weights

Answer 5

How much one neuron is firing to another (the lines that connect the units)

Question 6

Activation

Answer 6

If a neuron/unit is activated (or not)

Question 7

Analogue

Answer 7

On/off

Question 8

Parallel processing

Answer 8

Instead of having serial processing (one at a time) one has multiple at a time

Question 9

Back-propagation

Answer 9

Learning neural networks by gradient estimation (optimization algorithm)
The strategy exploits the calculated error between the actual values of the processing units in the output layer and the desired values, which is provided by a training signal. The resulting error signal is propagated from the output layer backward to the input layer and used to adjust each weight in the network. The network learns, as the weights are changed, to minimize the mean squared error over the training set of words

Question 10

Fundamental research program of classical AI

Answer 10

Identify the undoubtedly complex function that governs the human pattern of response to the environment and then write the program (the set of recursively applicable rules) by which the SM machine will compute it.

Question 11

Classical AI

Answer 11

It’s like a programmed robot following a strict set of human-made rules

Question 12

Anatomic points in the brain that inspired connectionism

Answer 12

Nervous system: Parallel machines
The neurons are comparatively simple: Analog response
- They are somewhat digital - it is firing or not
- We look at the firing frequency and translate it into a value between 0 and 1
- This is a contrast to computers 0’s and 1’s
In the brain, axons projecting from one neuronal population to another are often matched by axons returning from their target population

Question 13

Function (connectionism)

Answer 13

Any vector-to-vector transformation

Question 14

Materialism

Answer 14

Everything is physical

Question 15

Dualism

Answer 15

Distinction between body and mind

Question 16

NETtalk

Answer 16

It converts English text to speech sounds
1. Receives as input letter in a word
○ Local representation
2. Performs the transformation
3. Yield the elementary speech sounds
Has 309 processing units and 18,629 connection strengths (weights) that must be specified. The network does not have any initial or built-in organization for processing the input or (more exactly) mapping letters onto sounds. All the structure emerges during the training period. The values of the weights are determined by using the “back-propagation” learning algorithm developed by Rumelhart, Hinton, and Williams (1986)

Question 17

SM machine

Answer 17

Symbol manipulating machine

Question 18

Church’s thesis

Answer 18

Every effective computable functions is recursive computable

Question 19

Effectively computable

Answer 19

There is a “rote” procedure for determining, in finite time, the output of the function for a given input

Question 20

Recursively computable

Answer 20

There is a finite set of operations that can be applied to a given input, and then applied again and again to the successive results of such applications, to yield the function’s output in finite time

Question 21

The Luminous room (Maxwell)

Answer 21

Because something is valid it doesn’t need to be sound
Maxwell and light (theoretical)
Axiom 1. Electricity and magnetism are forces.
Axiom 2. The essential property of light is luminance.
Axiom 3. Forces by themselves are neither constitutive of nor sufficient for luminance.
Conclusion 1. Electricity and magnetism are neither constitutive of nor sufficient for light.
Problem: If you stand in a dark room with a magnet there will be no light
Maxwell’s responses:
The “luminous room” experiment is a misleading display of the phenomenon of luminance because the frequency of oscillation of the magnet is absurdly low, too low by a factor of 10^15.
But the room already contains everything essential to light according to Maxwell’s own theory

Question 22

Main problems with the Chinese rom

Answer 22

What people can or cannot imagine often has nothing to do with what is or is not the case, even where the people involved are highly intelligent
Axiom 3 is false

Question 23

The failure of classical AI

Answer 23

The functional architecture of classical SM machines is simply the wrong architecture for the very demanding jobs require

Question 24

Process training up network

Answer 24

There are various procedures for adjusting the weights so as to yield a network that computes almost any function-that is, any vector-to-vector transformation-that one might desire. In fact, one can even impose on it a function one is unable to specify, so long as one can supply a set of examples of the desired input-output pairs.

Question 25

Why is it important that the system is parallel?

Answer 25

• A parallel architecture provides a dramatic speed advantage over a conventional computer, for the many synapses at each level perform many small computations simultaneously instead of in laborious sequence.
• Massive parallelism means that the system is fault-tolerant and functionally persistent; the loss of a few connections, even quite a few, has a negligible effect on the character of the overall transformation performed by the surviving network.
• A parallel system stores large amounts of information in a distributed fashion, any part of which can be accessed in milliseconds

Question 26

Weight space

Answer 26

The space of all possible weights

Question 27

Naturalism

Answer 27

The idea that only natural laws and forces operate in the universe

Question 28

Mind-body problem

Answer 28

Are mental phenomena actually phenomena of the physical brain?

Question 29

Identity theory

Answer 29

Mental states are identical to states in the brain

Question 30

Reductions

Answer 30

Explanations of phenomena described by one theory in terms of the phenomena described by a more basic theory

Question 31

Reductionist

Answer 31

Integration of psychology and neurobiology theory

Question 32

Sententialism

Answer 32

The research paradigm that it is the task of cognitive science to figure out what programs the brain runs, and neuroscience can then check these top-down hypotheses against the wetware to see if they are generally possible

Question 33

Connectionist model

Answer 33

Characterized by connections and differential strengths of connection between processing units
Are designed to perform a task by specifying the architecture: the number of units, their arrangement in layers and columns, the patterns of connectivity, and the weight or strength of each connection

Question 34

Processing units (connectionist model)

Answer 34

Neurons that communicate with one another by signal (firing rate)

Question 35

Computational level (connectionist model)

Answer 35

The task is specified

Question 36

Implementation level (connectionist model)

Answer 36

The task is physically instantiated

Question 37

Booluean dream

Answer 37

(Old idea) all cognition is symbol-manipulation according to the rules of logic

Question 38

Neurobiologists’ dream

Answer 38

(Old idea) the faith that the answers we seek will be manifest once the fine-grain details of each neuron are revealed

Question 39

Marr’s Dream of three levels of explanation

Answer 39

In Marr’s view, a higher level was independent of the levels below it, and hence computational problems could be analyzed independently of an understanding of the algorithm that executes the computation, and the algorithmic problem could be solved independently of an understanding of the physical implementation
Marr’s assessment of the relations between levels has been reevaluated, and the dependence of higher levels on lower levels has come to be recognized
Marr’s three levels of analysis and the brain’s levels of organization do not appear to mesh in a very useful or satisfying manner

Question 40

Connectionist

Answer 40

Network models are not independent of either the computational level or the implementational level - maybe there are more than 3 levels (more than 1 implementational level)

Question 41

Tensions and shortcomings in computationalism

Answer 41

It is bound up on the research of AI
○ Example: Robot Shakey
§ Why is he so slow when computers usually are so fast?
§ 1) Signal propagation in the computer much faster than in the brain (~1e6 times faster)
The context problem

Question 42

The context problem (Dreyfus argument)

Answer 42

Humans: Have inarticulate background knowledge, which can be applied in a flexible manner to changing contexts
Classical artificial intelligence: Background knowledge needs to be stored explicitly. Access faces the double problem that: Search time is long, and what to search for (what is the context)?

Question 43

Why neural networks?

Answer 43

Reverse engineering the brain - and hopefully we will get the mind as well

Question 44

Redundancy (Fault tolerance)

Answer 44

The network doesn’t get down if we remove one piece

Question 45

Flexible storage

Answer 45

All knowledge is stored in the system

Question 46

Advantages of parallelism

Answer 46

Speed
Redundancy
Flexible storage

Question 47

Representation

Answer 47

Emerges from the structure of the neural network

Question 48

Emergent properties

Answer 48

A property of a complex system is said to be ‘emergent’ just in case, although it arises out of the properties and relations characterizing its simpler constituents, it is neither predictable from, nor reducible to, these lower level-characteristics.

Question 49

Question fallacy

Answer 49

An attempt to prove something is true while simultaneously taking that same thing for granted