Lecture 3 - The visual system

Question 1

The visual system

Answer 1

▪ The task of the visual system is to reliably extract meaning from noisy retinal input.

▪ In the matter of a second, your brain extracts information (such as segment, colour, category, inanimate or not, real-world size, type
of object and the implied action).

▪ From retina to the cortex:
- The light-sensitive photoreceptors in the retina transform light into action potentials and the neurons preform pre-processing.
- Information is relayed via the lateral geniculate
nucleus, a subcortical structure in the Thalamus.
- Information reaches the primary visual cortex (V1) in the calcarine sulcus, and also V2 and V3.

▪ Vision is lateralised. The two visual hemifields are split across the two hemispheres (cross-over).

▪ Vision is enabled by a network of cortical regions (>50% of cortex is driven by visual input).

Question 2

The early visual cortex (V1)

Answer 2

▪ Cortical representations are retinotopic and over-emphasise foveal input.
Receptive field sizes increase with eccentricity
(how far they are away from the center point).

▪ Hubel and Wiesel found that
V1, V2 and V3 all contain simple and complex cells:
- Simple cells: Excited by bars of light, oriented in particular directions. These cells behave like orientation detectors.
- Complex cells: Maximally excited by bars of light moving in a particular direction through the receptive field.

▪ Part of V1 response tuning can be described as complex edge detection.

▪ V1 can be understood as providing a sparse (overcomplete) representation of the outside world. A vector is sparse if most of its elements are close to zero, and a few of its elements are large in
magnitude. In a neural context, that means that a minority of
neurons are needed to encode a typical stimulus.

Question 3

The two streams hypothesis

Answer 3

▪ Dorsal visual pathway (where): Pathway that originates in the occipital cortex and projects to the parietal cortex.
It focuses on how action is to be guided toward objects.

▪ Ventral visual pathway (what): Pathway that originates in the occipital cortex and projects to the temporal cortex. It focuses on identifying objects.

▪ Koniocellular visual pathway: a lateral and third visual pathway found uniquely in humans and is likely related to emergent language function. Not much is known about this pathway.

▪ Along the posterior-anterior axis, latencies, receptive field sizes, and complexity increase.

Question 4

Other areas of the visual system

Answer 4

▪ Neurons in V2 exhibit more variety in feature selectivity going far beyond bars.

▪ V4 is sort of a jack of all trades. It is responsible for:
- Colour perception: viewing and separating colours
- Colour constancy: the perception that objects maintain a constant colour despite the fact that under different illumination conditions the wavelength composition of light reflected from the object changes significantly.
- Depth: binocular correspondence, size constancy.
- Shape: curvature, sparse coding of curvature.
- Intermediate complexity features/texture selectivity
(not yet complex object selectivity)
- Motion: motion contrast-defined shape
- Strong effects of feature-based attention and task.

▪ Tasks are often not exclusive to one particular region or area
(e.g. V1 also has data about colour perception).

Question 5

Facial recognition

Answer 5

▪ How can we tell to which features, cells in the Inferior Temporal cortex respond?
Exhaustive exploration of high-dimensional image space is not possible (as there are far too many options
to explore)!

▪ Faces are one of the most important categories, used
for social interaction, and identification.

▪ Macaque (type of primate) brains show a structured
face-patch system, with different parts that are
selective to different aspects of a face. ML shows a
preference for face viewpoints, AL shows mirror
symmetry, AM encodes the identity.

▪ Face patches emerge with experience, face selectivity is refined by smaller responses to objects.

▪ Having experience with faces is required for cortical face selectivity. Without experience with
faces, body selectivity takes over (such as hand selectivity in macaques). This does not prove
that face-selectivity is not innate, as it could be present at birth but then be lost.

▪ Plasticity (the ability to undergo structural changes) for acquiring face/non-face categorization
is preserved even later in life. Meaning that it is possible to learn how to discriminate faces at
later stages in life.

▪ Humans, too, show an extended network of face-selective areas:
- Fusiform Face Area (FFA)
- Occipital Face Area (OFA)
- Parahippocampal Place Area (PPA)
- Much is uncertain about face selectivity within
humans, as more certainty would require the
use of invasive methods such as single-cell
recordings (which would be unethical)
* Damage to these areas can lead
to prosopagnosia, which is the
inability to recognise faces.

▪ The extrastriate Body Area (EBA) responds to all forms of body depictions and body parts, but not to faces, animals or objects.

▪ Left-lateralized area activated for the visual display of full words.
This cannot be evolutionary. It emerges with reading acquisition and scales with reading proficiency.

▪ A theory for explaining the selectivity of brain areas could be that regions with foveal preference tend to be face-selective and peripheral selectivity goes with places.

▪ Selectivity of individual voxels is predicted by their connectivity to other regions all over the brain.

Question 6

Coding principles

Answer 6

How are conepts encoded in the visual system?

Scientist: Quiroga
Theory: 
single cell recordings in 
human medial temporal lobe 
reveal high selectivity.
Results:
- sparse, but no 
grandmother-cell
- implausible to have found 
the one cell out of a few 
hundred million neurons in 
the medio-temporal lobe. 
- theoretical considerations: 
each cell probably responds 
to ~50-150 individuals or 
objects.

Scientist: Haxby
Theory:
multivariate analysis of 
higher-level visual areas 
reveal a distributed code.
Results: 
- strong support for mainly 
face selective units in face 
patches, causal evidence from 
electrical stimulation of 
human FFA 
- computational models 
trained to recognise faces also 
allow to distinguish other 
concepts - likely because of 
overlapping feature selectivity.

▪ Personal view (thus, according to Dr. Kietzmann) : the system is distributed and sparse. Some
subsystems for ecologically relevant categories exist, likely due to other task constraints.