Object Recognition

Question 1

How is binding of features achieved in the visual areas?

Answer 1

There is some elementary binding by feedforward convergence (from low level features to higher level features (color, orientation etc, up to faces)) however most binding mediated by horizontal and feedback connections

Question 2

How do the functions of horizontal and feedback connections differ?

Answer 2

Horizontal connections link features processed within the same ares, Feedback connections link features processed across brain areas > full perceptual organisation.

Question 3

Describe two theoretical views of how the representation of objects is implemented in the brain

Answer 3

1) In the hierarchical model, there is feedforward convergence of hierarchal RF properties: from cells detecting low level features to cells detecting increasingly complex feature constellations , up to cells detecting highly specific objects: pontifical cells
2) In the model of dynamical assembly formation, cells each encode specific low and higher level features, but objects are encoded by the formation of dynamic assemblies (groups of cells) via horizontal and feedback connections, that together encode a specific object.

Question 4

How do the terms ‘grandmother cell’ and ‘yellow volkswagon cell’ ridicule the notion of this hierarchal model? (2)

Answer 4

Names for the pontifical cells

grandmother cell- you lose neurons daily, even more when you drink. What if you go on a three day bender, lose your grandmother cell and come home and smack the shit out this strange old lady in your home only to find out it’s grandma?

Yellow Volkswagon cell- For every possible object you need a specific cell which create a ‘combinational explosion.’ There are not enough neurons in the brain to represent every object.

Question 5

Which of these theories is backed by neurological evidence?

Answer 5

The visual brain seems to adopt both strategies: there is clear evidence for hierarchical processing, leading to ever more complex receptive field tuning, but there is also clear evidence for dynamic grouping via horizontal and feedback connections to ‘bring it all together’.

Question 6

What processes come the closes to the hierarchal model?

Answer 6

For very specific objects like faces, cardinal or even pontifical cells seem to exist.

For most other objects, however, there seems to be a need for assembly coding

Question 7

What’s a possible solution to the combinational explosion problem?

Answer 7

Ensemble coding theory- here your grandmother would be represented in the brain not by a single neuron but by a set of neurons , some encoding ‘faceness’, others hair colour, wrinkles, cheek colour etc. Their combined activation would then signal the presence of grandmother. Other combinations would signal other older ladies, or other people etc.

Question 8

How would this ensemble coding solution also combat the criticisms of the grandmother cell?

Answer 8

Ensemble coding is flexible, therefore more resistant to damage

Question 9

How is ensemble coding flexible?

Answer 9

1) Objects are represented by sets of neurons each representing individual features of the object
2) Individual ‘nodes’ may participate in different ensembles

Question 10

Name two issues which arise from ensemble coding theory?

Answer 10

How are assemblies labeled?

Does the brain really use two separate strategies? (hierarchal and assembly coding)

Question 11

In what way may assemblies be labelled?

Answer 11

Synchrony

Fire rate

Question 12

What is the need for a label?

Answer 12

There must be something to distinguish between assemblies

Question 13

Explain the synchrony label

Answer 13

Neurons of each assembly fire action potentials in synchrony. Assembly A and B code for different objects. The brain ‘knows’ which parts belongs together because those that belong to an assembly fire action potentials in synchrony.

Question 14

What evidence is there behind this synchrony label?

Answer 14

If two receptive fields are stimulated by the same object then it would result in synchrony. If two receptive fields are stimulated by a different object then there is not synchrony

Question 15

What evidence is there against the synchrony label?

Answer 15

Neural synchrony reflects orientation similarity but NOT figure-ground segmentation regardless of cue (e.g motion, cued by opposite petterns etc)

Question 16

What was a general consensus based on this data about synchrony?

Answer 16

Synchrony label may work as an initial process ‘laying down’ the assembly but the final process may be mediated by something else

Question 17

What may the final process of the synchrony assembly be mediated by

Answer 17

contextual modulation (cells responding to a figure rather than the background)- contrary to synchrony- DOES reflect figure-ground segregation regardless of cue. It reflects the final OUTCOME of perceptual organisation.

Question 18

Name a problem associated with this contextual modulation

Answer 18

How do we then distinguish between multiple objects

Question 19

What does the fire rate label mean and how does this contrast to the synchrony label irl

Answer 19

Simply increased activation and both labels seem to work together

Question 20

To what extent is it argued that face areas are involved in distinguishing between faces?

Answer 20

FFA/ Face neurons may encode the category for a face/ non face but the other visual areas may encode the individual features of a face required for individual face recognition.

Question 21

What evidence is there against this claim that FFA is just behind the categorisation of faces? (2)

Answer 21

face selective neurons first ‘recognise’ the category ‘face’ (at ~60ms), later on (~100 ms) recognise different faces or facial expressions

There are three face areas, why that much needed for only knowing whether it’s a face or non face

Question 22

Describe the study that Haxby presented in 2006 to try provide evidence for assembly coding in the human brain

Answer 22

Presented numerous objects from the same class and from different classes i.e faces, houses, shoes, cats, etc

He wanted to know whether these different classes are represented in the visual cortex and looked at patterns of fMRI activation in the visual cortex.

He presented these objects in several runs and correlated the fMRI patterns obtained from even and odd runs

Question 23

What were the results of the study that Haxby presented in 2006 to try provide evidence for assembly coding in the human brain?

Answer 23

Objects evoked widespread activity in temporal cortex (He only measured from the temporal cortex)

Each type of object evoked a typical pattern of activated voxels

The correlation between voxel patterns was higher between objects within the same category (r= 0.45 for chairs, 0.55 for shows) than between objects of different categories (r=-0.1)

Therefore voxel pattern is informative of object class

Question 24

What was observed when the maximally response voxels were removed from the analysis?

Answer 24

Also if the maximally response voxels (i.e. face selective voxels, house selective voxels, cat selective voxels, etc) were removed from the analysis, correlations between objects of the same category were still much higher than correlations between objects of different categories.

This implies that objects are not encoded by specific cells or regions, but by the distributed pattern of activity throughout visual / temporal cortex: assembly coding

Question 25

How was this assembly coding more directly assessed for faces?

Answer 25

First 8000 face are ‘decomposed’ into meaningful dimensions such as gender, expression, color, and all sorts of other (1024) ‘latent variables’. This can then be used to generate or modify faces (make them more smiling or male etc).

Then, a classifier system learned how the 1024 dimensions of the latent face space mapped onto the fMRI voxels of each subject (a). This system was then tested: show a novel face > record fMRI > reconstruct the shown face from this neural response (b).

This reconstruction was quite accurate (see examples for subject s1 to s4 in docs). The VAE-GAN version (ignore the PCA) was very good at pairwise recognition (is it face A or B?), highly above chance for full recognition (which of the 20 tested faces was it?)

Question 26

What important question was asked about this facial decomposition and reconstruction task regarding assembly coding? What was the answer?

Answer 26

Was the decoding depending on high- or low-level voxels? Only on FFA (temporal) or also on low levels (occipital)

Answer: Low level areas just as important > Face identity is encoded by assembly of low- and high-level neurons

Question 27

What two different streams of information can be discerned on the basis of anatomical connections?

Answer 27

The dorsal vs ventral streams

Question 28

Where do these streams run through on the rough diagram?

Answer 28

Dorsal stream
V1 > Superior longitudinal fasciculus, Posteroparietal cortex

Ventral stream
V1> inferior longitudinal fasciculus, Inferior temporal cortex

Question 29

How do these streams have different retinal inputs?

Answer 29

Dorsal pathway: Magnocellular dominated

Ventral pathway: Parvocellular dominated

Question 30

Why is there a distinction between the what and where pathways between dorsal and ventral stream?

Answer 30

Because the brain areas encoding motion and position (where) are typically along the dorsal pathway while the areas encoding colour and shape (what) are typically along the ventral pathway

Question 31

What experiments has the notion of these what and where pathways also been based on? (2)

Answer 31

Ungerleider & Mishkin experiment

Object (shape) discrimination was impaired with a lesion of the temporal lobes (couldn’t select foodwell with particular shape)

Landmark (position) discrimination was impaired when parietal lobes were lesioned. (couldn’t select foodwell closest to tower.)

Question 32

How else can these two pathways be defined in regards to the areas in which these streams end up in?

Answer 32

Dorsal- Motor, Vision for action

Ventral- Memory, vision for perception

Question 33

How can patients with agnosia demonstrate evidence for this dorsal and ventral theory

Answer 33

A patient with visual agnosia can recognise objects by touch, sound or smell. He knows the words, he can’t however recognise them by sight. Another example of the inability to recognise the object yet being able to express the action that goes along with it. (clarinet, lock)

Question 34

What is meant by associative agnosia?

Answer 34

Patients cannot name objects that they see (but can recognise what they feel, so they still know the name).

Question 35

How are people with associative agnosia at perceptual organisation or copying drawings?

Answer 35

Both fine, however they would not be able to name these objects

Question 36

What brain damage is associated with associative agnosia?

Answer 36

Mostly left (remember apperceptive is mostly right) hemisphere occipital/ parietal/ temporal lesions.

Question 37

Can associative agnosia patients match objects by function? (open closed umbrella etc)

Answer 37

Nah dawg, In associative agnosia, perceptual categories (organised percepts of objects) can no longer be coupled to semantic categories (non-visual knowledge about these objects)

Question 38

Describe the differences between patient DF (ventral lesion) and patient RV (dorsal lesion) and name their respective disorders

Answer 38

Patient DF (ventral lesion) cannot see shapes (optic agnosia), yet she picks them up normally Patient RV (dorsal lesion) sees the shapes, yet cannot pick them up normally (optic ataxia)

Question 39

What other task demonstrated the effects of optic agnosia? What were the results?

Answer 39

the SLOT task

DF cannot match orientation of bar with orientation of slot when only perceiving. Yet she ‘automatically’ orients the bar in the right orientation in visually guided action (posting)

Question 40

How is this different for an optic ataxia patient?

Answer 40

Optic ataxia patient (posterior parietal lesion) Can recognize orientation, but cannot ‘post’

Question 41

What concept do these SLOT tasks provide evidence for?

Answer 41

Combined with Milner and Goodale results provides evidence for a double dissociation between vision for action and vision for perception

Question 42

How is this concept of a double dissociation between action and memory different to what and where?

Answer 42

Dorsal stream: Vision for action–Uses position and shape information for visually guided action

Ventral stream: Vision for perception–Uses shape (and position) information for visual perception, memory

Question 43

How is vision for action versus perception in normal observers studied using an illusion?

Answer 43

Studying grip aperture immediately prior to picking up objects

Ebbinghaus illusions (smaller circles surrounding a circle makes it look bigger than bigger circles). They are perceptually different but are physically the same. It can also be don so that they are perceptually identical but physically different.

When the objects are perceptually identical but physically different we perceive them as the same size yet open our fingers wider to grip it. When they are perceptually different but are physically the same, you perceive them as different yet open your fingers the same amount to grip them.

Question 44

What experiment did Goodale carry out to examine these dualpathway visual processes in normal patients?

Answer 44

Goodale continued the experiments using a virtual workbench. Subjects were viewing objects via a mirror, that seemed to be lying below their hands. A size contrast illusion was applied so that two objects may have perceived bigger or smaller but were the same size in actuality.

The instructions were to pick up these objects, either after a delay or not, or indicate their sizes, using the same hand.

This way, both the perceptual judgment and the action were performed by the same hand, same ‘output’

Question 45

What were the results of Goodale’s experiment of these dualpathway visual processes?

Answer 45

It was found that when there was no delay between stimulus presentation and the grasping action, the hands made an equally large opening for the seemingly smaller and larger objects. Again visually guided action was not fooled by the size-contrast illusion.

However, when the action had to be performed after a memory delay, the hands were suffering from the size-contrast illusion

Question 46

What conclusions could be drawn from Goodale’s experiment of these dualpathway visual processes?

Answer 46

This difference in perception may only occur when the motor command is done almost automatically and not when done perceptually etc

Question 47

What was the second part of Goodale’s experiment of these dualpathway visual processes and what were the results?

Answer 47

In a second experiment, the task of the subject was to indicate the remembered size of the object (with the red dot) that had appearedIn this case, whether with a delay of 5 seconds or no delay, the size estimation suffered from the size-contrast illusion

Question 48

To summarise, what did Goodale’s experiment of these dualpathway visual processes demonstrate? (3)

Answer 48

• Perception (size estimation) suffers from visual illusions
• Memory guided (delayed grasping) action suffers from visual illusions
• ‘Online’, immediate visually guided action does not suffer from visual illusions

These effects are independent of the mode of response (fingers)

Supporting the notion of separate ‘vision for action’ and ‘vision for perception’ pathways, also in normal observers

Question 49

What explanation could there be for this discrepancy between perception and immediate visually guided action?

Answer 49

At short distance (within reach of hands) we can use stereopsis and disparity to judge distance and size.At longer distances we cannot use disparity (because for far away vision retinal images are virtually the same). There we have to rely on relative size and other (monocular) cues of depth, distance and size. These are useful mechanisms, that however also give rise to illusions (slide 12, 13, 57). Objects in the distance are not dealt with immediately, but after some delay, or have to be memorised. This is when perception starts to influence action.

The vision for action system is the connection between eyes and hands > immediate visually guided action

The vision for perception system is the connection between eyes and later actions > memory