Chapter 4 Flashcards
What are the problems connectionists have with classical and how do they address them?
Classical models invoke serial processes, which make them too slow to run on sluggish componentry (like the brain); they involve explicit, local, and digital representations of both rules and symbols, making the models too brittle. As a result, classical models do not degrade gracefully. Essentially, these models are too based on the metaphor of a digital computer, and do not show biological plausibility.
Connectionists offer new models, with the basic medium a type of model called the artificial neural network, or a parallel distributed processing network (PDP). Artificial networks consist of a number of simple processors that perform basic calculations and communicate the results to other processors by sending signals through weighted connections; this runs in parallel, permitting fast computing even when slow componentry is involved; these models exploit implicit, distributed, and redundant representations, making the networks not brittle, and can degrade gracefully; due to the nature of neural networks (intentionally inspired by the neuronal network of the brain) they are biologically plausible.
Describe the behaviour of a processor in an ANN
- the processor computes the total signal (its net input) being sent to it by other processors in the network.
- the unit uses an activation function to convert its net input into internal activity (usually a continuous number between 0 and 1) on the basis of this computed signal
- the unit converts its internal activity into an output signal, and sends this signal on to other processors
» a network uses parallel processing because many, if not all, of its units will perform their operations simultaneously.
What elements of empiricism do ANN evoke?
- the notion of a ‘blank slate’; the mind is blank in the absence of experience (think of all the connection weights being assigned random values, are equaling zero)
- the source of ideas or knowledge or structure is experience
What is a Kohonen network?
A famous kind of self-organizing network, where output units are arranged in a two-dimensional grid; unsupervised learning causes the grid to organize itself into a map the reveals the discovered structure of the inputs, where related patterns produce neighbouring activity in the output map
What is the delta rule?
gradient descent learning rule for updating the weights of the inputs to artificial neurons in single-layer neural network
What were Aristotle’s three laws of association?
- similarity
- opposition
- (temporal) contiguity
What does James’ biological account of association reveal about properties which are common to modern connectionists?
- his system was parallel
- his system was convergent
- his system was distributed
What is a standard pattern associater?
A modern associative memory system; structurally identical to a distributed memory (neuron group A [a –> e] connected through modifiable connections to neuron group B [l —> p], which can essentially store information in these connections) it is a memory capable of learning associations between pairs of input patterns or learning to associate an input pattern with a categorizing response
What are hidden units?
The solution to this problem is to add layers of intermediate processors, called hidden units. The result is the multilayer perceptron. Hidden units also use nonlinear activation functions. This permits a sequence of
complex decisions to be made by an artificial neural network. This required new error-driven learning rules to be discovered, which only happened around
the mid 1980s
What is the all-or-none law?
How a neuron transforms graded potentials into AP’s (which are all or none, either on or off); one is generated and it is always full size, minimizing the possibility that information will be lose along the way
What is the McCulloch-pits neuron?
Using the all-or-none law to justify describing neurons very abstractly as devices that made true or false logical assertions about input information. It is a connectionist processor; it is very similar to the standard pattern associator except that once the net input is computed (by summation of all of its incoming signals) the ‘neuron’ uses a nonlinear activation function (the Heaviside step function) to transform net input into internal activity.
What is a perceptron?
Artificial neural networks that can be trained to be pattern classifiers; given an input pattern, they would use their nonlinear outputs to decide whether or not to the pattern belonged to a particular class.
» aka they could assign perceptual predicates, something standard pattern associators can’t
What is an arbitrary pattern classifier?
An arbitrary pattern classifier would be capable of solving any pattern recognition problem, which would mean that it could carve any pattern space into any set of decision regions that could ever be needed.
What is a universal function approximator?
A universal function approximator is a system that, given a set of predictor variables, can output an accurate estimate of some predicted variable.
» for a function approximator to be universal, it must be able to compute the value of any function to an arbitrary degree of precision, at least in some range of the function.
What is distributed representation?
A distributed representation is a concept that is central to connectionism. In a connectionist network, a distributed representation occurs when some concept or meaning is represented by the network, but that meaning is represented by a pattern of activity across a number of processing units (Hinton et al, 1986). In other words, the meaning is not locally represented by a single unit that is analogous to a “grandmother cell”.
One advantage of distributed representations is that they provide damage resistance and graceful degradation (Medler et al., 2005). A disadvantage of such representations is that they make the internal structure of a trained network very difficult to interpret (Dawson, 2004).
