Low Level Visual Processing

Question 1

structures of the eye

Answer 1

• Complex structure made up of different parts
• Cornea: focuses light, fixed thing on the eye
• Pupil: the aperture in the iris, changes size according to light levels
  
  • Inside, beneath the cornea
• Lens: focuses light, adjustable ——> accommodation
  
  • Not fixed (held in place by muscle that helps the lens accommodate- seeing nearer and far and change between this quickly)
  • When you get older the muscles slacken and it becomes difficult for the lens to accommodate changes in distances- glasses help with this
• Retina: rods and cones are cells in the retina that convert light into an electrochemical signal
  
  • Sit around the body called the fovea (dip in the back of the eye)
    Light hits the retina which activates the rods and the cones which converts it into the electrochemcial signal which is sent to the brain via optic nerve

Question 2

structure of the retina

Answer 2

• The retina is filled with light sensitive rods and cones- covert light into electrical activity
• 125 million rods in each eye, 6 million cones
  Nerve fibres come down from the ganglion cells- axons of neurons. They pool all the information coming from rods and cones and they bundle it down the optic nerve

Question 3

how the rod and cone receptors send signals

Answer 3

• When enough light falls on a rod or cone cell in the retina, that cell sends an electrical signal down its nerve fibre (an axon).
• This signal is like an action potential. It triggers a chemical signal to another neuron which, in turn, can send signals to other neurons.
  The rods and cones convert light into neural signals - the language of the brain.

Question 4

the electromagnetic spectrum

Answer 4

• The electromagnetic spectrum - visible light is only a tiny part of it
• nm = nanometres = 10-9 metres or one billionth of a metre, so very, very small; 1000 nm = one millionth of a metre
• Range of energy- the bit in the middle is the only bit that is visible (visible light)
  
  • That reaches between 380-700nm which is how this light is measured
  • All the light at very short wavelengths are on the left (blue section)
    Our vision has been adapted through evolution to pick up visible light- different receptors in our eyes pick up different parts of visible light

Question 5

rods and cones

Answer 5

• Rods and cones have similar sort of structure
• Rods help us see at night- however they are not good acuity (cannot see things sharply, no colour vision)
  
  • Rods help us navigate the world at night (no colour just identification of objects)
• Cones operate during time- give us colour vision
  
  • 3 different types of cones that all correspond to different parts of the EM spectrum- red cones (long wavelength), green cones (medium wavelength) and blue cones (short wavelength)
    Also responsible for spatial acuity

Question 6

cones

Answer 6

• mostly in the fovea
  
  • Fovea at the centre of the retina (where acuity is the highest). As you move away from the fovea the acuity decreases
• are short, middle and long wavelength cones, which detect different colours.
• used in normal lighting conditions and in most situations. We move our eyes to bring important things onto the fovea.
  The fovea is high acuity as outputs from only a few cone cells are pooled together

Question 7

rods

Answer 7

• mostly in the periphery, outside the fovea.
• specialized for vision under low light conditions,
• do not distinguish between different colours (check for yourself at night). The periphery is low acuity as outputs from hundreds of rod cells are pooled together.
• Rods have low acuity but need to foveate with these- eyes continuously move to centre what we are looking at onto our fovea
  Periphery has very low spatial acuity

Question 8

the eye

Answer 8

• If you focus on the F spot you cannot see the black spot
  
  • Blind spot- so we cannot see the black dot as our brain fills in the gaps
  • Depending on the distance and where we are sat
• Letters in the periphery have to be much larger by a factor of 10 to give us the same spatial resolution as the middle as acuity is so low
  This is all processed by the rods

Question 9

from the eye to the cortex

Answer 9

• Most of the back of the brain (‘posterior regions’) are dedicated to visual processing.
• Nerve fibres go from the eye to the Lateral Geniculate Nucleus in the Thalamus (LGN), and then to the primary visual cortex (V1)
  
  • LGN sits above your ears and is where the visual brain sits- info leaves the eye via the LGN towards the visual cortex
  • Laterality of vision- one side controls the other
  • Each type of photoreceptors travel down different paths to the brain
• The right visual field projects to the left cerebral hemisphere,
• and the left visual field projects to the right cerebral hemisphere
• Neurons in V1 have receptive fields: They fire when stimulus is present in a particular part of the visual field.
• Retinotopic organization in V1 Neurons with nearby receptive fields are anatomically close to each other in V1.
• Cone cells (L and M) contribute to the Parvocellular pathway. Most sensitive to color and fine detail.
• Cones and Rod cells contribute to the Magnocellular pathway, which is most sensitive to luminance (changes in light) and motion (evolved to stop attacks from behind) in the periphery.
• Cones (S and L,M) feed into the koniocellular pathway, which mediates mostly colour vision (Yellow-blue pathway)
• As we will see, the distinction between parvo and magnocellular pathways continues up the visual hierarchy, beyond V1.
• V1 is the primary visual cortex- everything goes here
• The visual brain is divided into different anatomical regions, with each area having a distinct function.
• Most cells in V1 and V2 only detect simple information (eg oriented lines) whereas cells at later stages may, for example, respond to complex shapes and motion patterns:
• V4 =Colour V5=Motion LO = Shape
• Processing cascades up through the different areas of the brain, starting from the LGN and then to the visual cortex (V1, V2, V3, V5 and V5) and to IT (inferotemporal cortex)
• The size of the boxes represents the rough size of each brain area whilst the percentages show the proportion of nerve fibres in each pathway
  Happens in a short space of time

Question 10

how do we know anything about the visual system? some methods…

Answer 10

• We can disrupt processing in brain areas with Transcranial Magnetic Stimulation (TMS)
• We can record brain activity using functional Magnetic Resonance Imaging (fMRI).
• We can measure the effect of accidental damage to specific brain regions in humans.
• We can probe individual neurons in live animal brains with microelectrodes. When the ‘preferred’ stimulus is presented, the neurons fire.
• When it comes to the visual system, all these techniques provide converging evidence about the specialization of function
• With Transcranial Magnetic Stimulation, we can temporarily disrupt neural processing in specific visual map
• TMS to V5 produces a sense of motion
• TMS to V4 produces coloured phosphines.
• Function Magnetic Resonance Imaging (fMRI)
• Brain lesions = holes in the brain caused by accidents, strokes, tumours etc.
• Lesions often have specific effects, loss of language, difficulty seeing faces.
• Some lesions can change personality dramatically (As in the famous case of Phineas Gage, see images to right).
• Akinetopsia means difficulty seeing motion - Caused by discrete damage to area MT (also known as V5).
• Below is a description of a patient with Akinetopsia
• “She had difficulty .. In pouring tea or coffee into a cup because the fluid appeared to be frozen, like a glacier. In addition, she could not stop pouring at the right time since she was unable to perceive the movement in the cup or pot when the fluid rose”.
• In a room where more than two people were walking… “people were suddenly here or there but I have not seen them moving.”
• Zihl et al. (1983).
• Damage to area MST selectively impairs optic flow processing which we use for walking (Vaina, 1998).
• First order motion and second order motion are different. First order motion = luminance defined motion. Second order motion = Contrast defined motion.- do not need to know this for the exam
• Cells in V4 fire are to some extent selective to colours (but broadly tuned)
• There are no neurons that are only driven by red, yellow, green etc
• Bouvier and Engel (2006) ran a “meta analysis” of all lesions which create achromatopsia (trouble seeing colours). The most common lesion site was V4.
• Some colour processing is spared after V4 lesions –so this is not the only colour center in the brain

Question 11

micro electrode recording

Answer 11

• We can actually pick up various types of neural activity with microelectrodes.
• Most simply, we can measure action potentials = neurons firing electrical pulses down the long axon.
• Firing rate increases when the neuron is active
  When we say a neuron ‘responds’ to something, we mean there is an increase in firing rate when an animal looks at one stimulus, but not at another

Question 12

from eye to cortex 2

Answer 12

• In the inferior temporal cortex, there are neurons that fire when we see certain shapes but not others.
• A cell might fire very strongly when a monkey is looking at a star, but not at a circle.
• The “binding problem”
  
  • How to we integrate information e.g. how do we know the circle and the square should be different colours
  • Best guess in recent years is called the synchrony theory- how we integrate information between shape an colour is all the neurons switch between intervals of low and high oscillation when something is stimulating them- when they code between singular features they oscillate in synchrony together so you get a synchronised assembly of thousands of neurons
  • You then get an asynchrony to integrate these features
  • Neurons respond to shape and other ones respond to colour oscillate at different times and bind together to integrate the features to detect these things quicker
• If we had different groups of neurons responding to different features, and these neurons are in different neuroanatomical regions, how we group the features which belong to a particular object.
• We have colour cells in V4 and shape cells in the inferotemporal cortex.
• How do we know that the blue goes with the circle shape, yellow goes with the star, and red goes with the square?
• Neurons switch between intervals of low and high excitability rapidly. All neurons that code features of a single object oscillate in synchrony (Singer and Gray, 1995). There is a synchronized assembly of thousands of neurons.
• But how does the brain synchronize the right neurons?
• When problems like this start to look difficult, you are mastering vision science! We take vision for granted. But it is deeply mysterious.

Question 13

ventral and dorsal stream

Answer 13

• Ventral stream is also called the ‘what stream’. It is involved in identifying objects.
• Is it a car? Is it a plane?
• The Dorsal stream is called the ‘where stream’.
• It controls actions.
• Milner and Goodale (1995) are famous for work on the dissociation between ventral and dorsal streams:
• Double dissociation: Damage to ventral stream should impair object recognition but not action planning, while damage to dorsal stream should impair action planning but not object recognition.
• Optic ataxia: damage to the posterior parietal lobe (part of the dorsal stream).
• Patient profile partially fits with the theory as they have some difficult action planning. E.g., inserting a card through an oriented slot (although they are not a simple group). Can label objects reasonably well.
• Visual form agnosia patients have the opposite problem. DF cannot name drawings of objects. However, she is very good at in inserting a card in a slot.
• BUT Optic ataxia and visual form agnosia patients are not the perfect double dissociation the theory demands. Both have some impairments both with action planning and object recognition.
• Schenk and McIntosh, 2010 list the four characteristics of the two processing stream
  
  • The ventral stream underlies vision for perception, while the dorsal stream underlies vision for action
  • Coding in the ventral stream is allocentric (object-centered, independent of observers perspective), while coding in the dorsal stream is egocentric (body-centered, dependent on the observers perspective)
  • Representations in the ventral stream are sustained over time, representations in the dorsal stream are short-lasting
  • Processing in the ventral stream typically (but by no means always) leads to conscious awareness, whereas processing in the dorsal stream does no
• Visual illusions might be unique to ventral stream
• When people grab the central circle quickly, they supposedly used the dorsal stream. On average, grip aperture is only 5.5% wider when approaching circle 2 than circle 1
• When people verbally estimate the diameter of the central circle, they supposedly use the ventral stream. The illusion is 22%.
• Q. Why is the illusion zero in the grabbing and pointing tasks?
  If the action is slow and measured, the effect of the illusion is larger. It is likely that representations in the ventral stream can be used to guide some types of more deliberate action.

Question 14

colour vision

Answer 14

• Metameters- two colours appear to match under one light source but not another
  
  • Common for neutral colours like grey and white- problematic for fashion designers
• The human retina contains only 3 kind of cone cells (L, M, S), Human colour vision is trichromatic. The cones form specific excitatory and inhibitory connections on bipolar cells.
• One kind of bipolar cell is excited by red cones, but inhibited by green cones.
• If faced with a white patch, the bipolar cell fires at a baseline rate. If faced with a red patch, the cell is excited, and fires rapidly.
• If faced with a green patch, it is inhibited, and fires at below baseline rate.
  Another is the opposite way round (excited by green and inhibited by red).

Question 15

colour vision: opponent colours

Answer 15

• In the brain the colour signals are recombined into opponent channels:
• RED-GREEN BLUE=YELLOW
• But there is no ‘greenish red’ or ’bluish yellow’. Why?
• This is because of colour opponent channels.
  When we stare at a colour for a long time, the response of cones fatigues. When we look at a white area, the opposite colour dominates. This gives us colored after images.

Question 16

colour constancy

Answer 16

• Colour constancy
• How can your brain tell that things are the same colour when the lighting conditions vary dramatically?
• The wavelengths reflected from objects are very different in the morning and the evening, but the colour looks about the same.
• Likewise, some artificial illuminants are yellow, others are white. This doesn’t have a dramatic effect on our sense of object colour.
• Chromatic adaptation: We slowly adjust to stable properties of light. Electric light doesn’t look yellow after a while.
• When problems like colour constancy start to look difficult, you are mastering vision science! We take vision for granted. But it is deeply mysterious.
• Part of the story is cone excitation ratios.
• Imagine you look at a mug on a table:
• At dusk, Surface 1 makes red cones fire 1 time per second, + Surface 2 makes them fire 3 times per second.
• In the day, surface 1 makes the red cones fire 10 times per second and surface 2 makes them fire 30 times per second.
• The ratio is 1:3 both at night and at day.
  We still perceive the same colour, because your visual system makes use of relative excitation across different surfaces.

Question 17

depth perception

Answer 17

• The retinal image is 2D – it doesn’t say how far the light has travelled before hitting a photoreceptor
• Monocular or pictorial cues are those we can use use when one eye is shut
• Linear perspective: Parallel lines converge on the horizon.
• Texture gradients: These can be perceived as a surface receding into the distance. The finer texture is further away.
• Occlusion or interposition. Near things occlude far things.
• Familiar size: If we have a sense of typical size, and it looks small on the retina, we can assume its far away (Ittelson, 1951).
• Motion parallax – if I move my head, everything moves across the retina. Near things move further than far things.

Question 18

depth perception- binocular and oculomotor cues

Answer 18

• Vergence (if we are looking at something really close, our eyes turn inwards)
• Accommodation (the lens changes shape to focus on near and far things, once something is in focus. Lens shape tells us the distance of the object in focus.
• Both vergence and accommodation are very limited (can only tell us about the position of a single object)
• Binocular disparity – the image on each eye is slightly different. The nearer something is the greater the difference in retinal position on each eye.
• 3D cinema works by presenting slightly different images to each eye. This ‘really looks 3D’.

Question 19

depth perception- cue combination

Answer 19

• Usually there are a lot of different depth cues available – How does the brain integrate them?
• Average estimate based on all cues? Always rely on just one of the cues?
• What we actually do is something like weighted average (Jacobs, 2002). We put more weight on the more reliable cues, and reliability estimates can be updated. Note how effortless the visual system does this this complex operation.
  How do we actually make a weighted average estimate out of neurons? How would you wire up a load of neurons to do mathematical operations like this? When problems like this start to look difficult, you are mastering vision science! We take vision for granted. But it is deeply mysterious.

Question 20

the physics of light

Answer 20

• In physics colour is associated specifically with electromagnetic radiation of a certain range of wavelengths visible to the human eye. These constitutes a portion of the electromagnetic spectrum known as the visible spectrum—i.e., light.
  
  • Before Isaac Newton’s experiment the notion was that light was ‘white’, the reasoning for refracted light,(when colours that appear when light penetrates glass ), was that it became contaminated as it travelled through any material.
    Newton’s experiment was important because it demonstrated this belief was wrong by showing that refracted light could be concentrated back into white light.

Question 21

light

Answer 21

• Newton came up with the idea of making a tiny hole in a piece of wood to isolate a small ray of light. He then directed that light into a prism that refracted the light onto a white piece of paper.
  
  • -he was able to identify different colours, seeing red and blue at the edges of where the ray was hitting the paper.
  • Isaac Newton’s light experiment involved the use of 2 prisms into the experiment.
  • directed the light that came out of the first prism into a second prism which produced white light again.
  • able to conclude that while the first prism had refracted the light, the second prism had concentrated the rays back together, and therefore it appeared as white light.
    In other words, the colours that appeared when light was refracted through a prism were proven to naturally be present, rather than a result of contamination.

Question 22

visible light

Answer 22

• Visible light corresponds to a small range of the electromagetic spectrum roughly from 400 nm (which appears blue) to 700 nm (appears red) in wavelength.
  Visible light is the portion of the electromagnetic spectrum visible to the human eye. It ranges from the red end to the violet end of the spectrum, with wavelengths from 700 to 400 nanometres

Question 23

colour deficiencies

Answer 23

• Two kind of deficits: Retinal (one or more of the three kinds of cones is missing)
• Protanopia: L missing -> you can’t distinguish red/green
• Deuteranopia: M missing -> you can’t distinguish red/green
• Tritanopia: S missing -> you can’t distinguish blue/yellow
• About 5-10% of individuals have colour deficiencies
• Mostly males because of the link with the Y chromosome. Deuteranopia is the most common case.
• Many forms of deficiency are consistent with trichromacy theory
• Retinal- retinal cells or cones
• Most people with colour deficiency have dichromacy in which one type of cone is missing
• In protonopia makes red look more green
• Deuteranopia- green looks more red
• More cones in the fovea than in the periphery- however there are usually enough cones in the periphery to allow fairly accurate peripheral colour judgements
• Cerebral: the damage is to the cortex, more specifically area V4, in the inferior part of the occipital lobe, in the lingual and fusiform gyri (Zeki, 1973).
• Cerebral Achromatopsia: difficulty in perceiving colours, world is in shades of grey
  Cerebral Hemiachromatopsia: relatively normal vision in one half of the visual field, but sees the other half in shades of grey

Question 24

chromatic pathways

Answer 24

• This is a schematic diagram of the stages of the neural colour processing pathway
• Hurvich and Jameson (1957): Dual process theory; signals from the 3 cone types identified by trichromacy theory are sent to the opponent cells
• There are 3 channels
  
  • The achromatic (non-colour) channel combines the activity of the medium/long wavelength cones
  • 2. The blue-yellow channel represents difference between the sum of the medium and long wavelength cones on the one hand and the short wavelength cones on the other. The direction of difference determines whether blue or yellow is seen.
  • 3. The red-green channel represents the difference between activity levels in the medium and long wavelength cones- again the direction of the difference determines whether red or green is seen.
• 3 cone classes (red= long, green= medium, blue=short) supply 3 channels
• The achromatic (light dark) channel receives non spectrally opponent input from long and medium cones. The two chromatic channels receive spectrally opponent inputs to create
  non-spectral hue is a hue which is not present in the spectrum of colours produced by splitting white light with a prism. Non-spectral hues include brown and the pastel colours.

Question 25

2 streams from eye to cortex

Answer 25

• The magnocellular and parvocellular pathways (M and P pathways) are the main pathways of the visual system, accounting for most of the axons that leave the retina and the perceived vision.
• Parvocellular neurons are sensitive to colour,- more capable of discriminating fine details than M-cells. P- cells have greater spatial resolution, but lower temporal resolution, than the M- cells.
• M-cells are neurons located within the magnocellular layer of the LGN of thethalamus. Name relates to their relative size compared to P-cells
• The M-cell pathway doesn’tprovide finely detailed or colour information, but useful static, depth, and motion information
• The M pathway has high light/dark contrast detection and is more sensitive at low spatial frequencies -essential for detecting changes in luminance , visual search , detecting edges and location of objects. M cells can detect the orientation and position of objects in space- information that is sent through the dorsal stream. This information is also useful for detecting the difference in positions of objects on the retina of each eye, an important tool in binocular depth perception
• Cells in the M pathway have the ability to detect high temporal frequencies and can thus detect quick changes in the position of an object
  Some information has also been found to support the hypothesis that the M pathway is necessary for face processing

Question 26

opponent colours

Answer 26

• The idea was first proposed by Ewald Hering in 1878.
• We are not sure about the reason for this hard-wiring, but there are some suggestions from information theory about matching internal dimensions to the main factors that describe light on our planet.
  So, if we see a red object, it is not because the longwave-sensitive (‘red’) cone sends a message, it is because a nerve cell that COMPARES the simulation of the red and green cones send a message saying: “more longwave than mediumwave light”.

Question 27

hue, saturation and brightness

Answer 27

• Hue is what we could loosely refer to as colour, as in red, yellow, blue etc. Hueless or achromatic colours include black, gray, white.
• Saturation refers to purity, or how much grey there is in a colour. Pink for instance differs from Red mainly because of saturation. Brown also tends to be unsaturated.
• Brightness goes from complete dark to dazzling, self-luminous white.
  These are three dimensions derived from people’s judgements of colours. They are the three dimensions of the human colour space.

Question 28

metameters

Answer 28

• A metamer or ‘match’ happens when two colours that are not actually the same colour appear the same under certain lighting conditions.
• Occurs because the colours have different electromagnetic spectrum distributions -this causes them to reflect contrasting wavelengths of light.
• The proportion of total light given off by a colour sample at each visible wavelength; defines the information about the light coming from the colour sample.
• BUT the human eye contains only 3 types of cones), which means that all colours are reduced to three sensory quantities, called TRISTIMULUS VALUES
  Metamers occur because each type of cone responds to the summed-up energy from a broad range of wavelengths, so that different combinations of light across all wavelengths can produce equivalent receptor response(s) and the same tristimulus values (or colour sensation).

Question 29

simultaneous colour contrast

Answer 29

-look at the X’s, they both look as though they are different colours. However, they’re not. The colours are exactly the same tone:
- Look at where both the X’s intersect at the middle-base of the image- the ‘true colour’ is in fact a greyish-yellow tone (a mixture of the two colours used in the rectangles). This effect is based on a painting by Josef Albers.
- This effect is know as trichromacy and the images above demonstrate simultaneous contrast
- The X on the left is surrounded by yellow. The X on the right is surrounded by gray.
- Simply-our eyes record colour in RGB—but none of us can image a yellowish-blue.
- The paint/pigment of the two X’s is identical, yet the colour appearance is different because the surrounding context is different.
Color perception, like brightness perception, depends on contrast/surrounding context. Our brains therefore plays tricks on us, resulting in the effect/illusion.

Question 30

colour assimilation

Answer 30

• Color assimilation occurs when two colour patches are perceptually grouped, and their apparent colour difference is reduced.
• King (1988) hypothesized that contrast occurs when patches are perceived as separate “wholes” while assimilation resulted when the two patches are perceived as one integrated “whole.”
• 2 At a higher spatial frequency, colour contrast is replaced with assimilation. The squares in the bottom and upper are identical, Both sets of squares begin to take on the colour of the squares they are placed next to.
• example of the assimilation or “spreading” effect
• .
• The green or white (depending on the diagonal) areas adjacent to the red squares influence the appearance of the red squares. The white make the red appear lighter and the green make the red appear darker. This is because the green is a much darker colour than the white. You may also note that the green casts a slight brownish tinge on the red squares. It is as if a blending of colours occurs.
• This phenomenon is the opposite of a contrast effect (where nearby colours should accentuate the differences between adjacent areas).
  Hurvich and Jameson believe that the physiological basis for the assimilation phenomenon is the spatial/cellular organisation- the way the cones and rods are distributed across the retina.

Question 31

Munker illusion

Answer 31

• The Munker or Munker-White illusion is an optical illusion illustrating the way that the same target luminance can elicit different perceptions of brightness in different contexts
• Munker’s illusion is based on something called the von Bezold effect In which a colour may appear different depending on its relation to adjacent colours.
• It happens when small areas of colour are interspersed. An assimilation (like the one seen earlier) effect called the von Bezold spreading effect is achieved.
  The opposite effect is observed when large areas of colour are placed adjacent to each other, resulting in colour contrast

Question 32

colour constancy

Answer 32

• Colour constancy is the tendency for a surface or object to be perceived as having the same colour when ther are changes in the wavelengths contained in the light source that illuminates the object or surface
• Colour constancy indicates that colour vision does not solely depend on the wavelengths of light reflected from an object’s surface
• If we didn’t have it the apparent colour of familiar objects would change dramatically under different lighting conditions- it would be really difficult to recognise objects as rapidly and accurately as we do
  Color constancy can be linked to chromatic adaptation wherein the visual system adjusts its sensitivity to a light according to the context in which the light appears

Question 33

colour vision

Answer 33

• Colour vision in non-humans
• Many vertebrates have four types of cones: red, green, blue and one type sensitive to the ultraviolet wavelengths (to which humans are blind).
• So, as well as seeing differences between colours that look identical to us they will see hues we cannot imagine.
• Most mammals have only two types of colour receptors in their eyes, one for blue (peaking around a wavelength of 445 nanometers) and one for yellow (555 nm).
• The ancestor of our particular branch of apes apparently experienced an accidental gene duplication, giving them three types of receptors: blue and yellow and yellow.
  Over time, it became advantageous for the two yellow types to diverge slightly, specializing so that one peaked at 535 nm (green) and the other at 575 (red). Now these apes could distinguish those two colours — as can most modern humans.