Week 3: feedforward visual processing

Question 1

Why do we study vision networks to understand DCNN?

Answer 1

• image processing is a common application of artificial deep networks
• the early visual system is the best understood system in the human brain
• early visual responses are a common application of DCNN as simulations of neural processing

Question 2

Where do responses to specific edge orientations emerge?

Answer 2

V1, the primary visual cortex. first area processing vision in the brain

Question 3

How is the first step of visual processing different in biological networks and artificial networks?

Answer 3

In biological network, contrast is initially computed in an orientation-independent filter in the retinal ganglion cell. Artificial DCNNs often skip this and go directly to edge orientation.

Question 4

What happens in V1?

Answer 4

Orientation-selective responses are computed in V1 by operations comparing the outputs of retinal ganglion cells

Question 5

How are neurons processing the image grouped?

Answer 5

Neurons processing the same part of the visual field are grouped, and neurons with similar orientation preferences are grouped, this grouping is at a very small scale.

Question 6

How do neurons have different orientation preferences?

Answer 6

Neurons have orientation preferences which gradually change across the cortex

Question 7

How do orientation preferences form feature maps?

Answer 7

The orientation columns form further feature maps which are squeezed into the same 2D cortical surface

Question 8

What are in the feature maps at V1?

Answer 8

In V1 there is a large complex set of feature maps with each feature represented at all spatial positions. Includes colour, eye, spatial frequency, orientation, motion direction.

Question 9

What happens after V1 in terms of processing an image?

Answer 9

Form (object recognition) and motion (motion and space) information are processed separately in different areas of the brain. There are multiple branching hierarchies performing different tasks

Question 10

How does the hierarchy of the brain differ to that of a neural network?

Answer 10

Lots of brain areas sample from V1 creating a web of connections. Artificial networks use a linear hierarchy

Question 11

What happens as you go up the hierarchy V1-V4… in the brain?

Answer 11

As you go up, the areas have a larger representation of the central visual field, and respond to increasingly complex features

Question 12

What happens to the spatial integration of the image as you move up the hierarchy?

Answer 12

Spatial relationships between image locations are maintained

Question 13

How does V1 represent the different areas in the visual field?

Answer 13

V1 strongly over-represents the central visual field compared to the peripheral parts of the field

Question 14

Which parts of the brain focus on object recognition and spatial perception?

Answer 14

Object recognition: ventral stream, temporal lobe

Spatial perception/action planning: dorsal stream, pariatal lobe

Question 15

How do later visual field maps sample from earlier visual field maps?

Answer 15

Later visual field maps sample from approximately constant cortical areas of earlier visual field maps, regardless of the visual position represented

Question 16

How does an artificial network filter represent a sample from the visual field map and how is it different?

Answer 16

Later visual field maps sample from constant cortical areas of earlier visual field maps. this is represented by the fixed size of artificial network filters
Differences:
-the inputs in biological networks over-represent central vision
-the input in biological network is neurons rather than images

Question 17

How do features transform as they move up through the layers of the hierarchy?

Answer 17

Transformations find commonly-seen patterns in activity of earlier layers. this has been difficult for humans to recognise

Question 18

What types of computations are later stages in the hierarchy doing?

Answer 18

Later stages are likely doing the same computations as earlier stages, but from more abstracted inputs

Question 19

Why do we use DCNNs to simulate feature transformations?

Answer 19

It is hard for humans to think about the transformations and representations of later layers in the network. DCNNs are useful for experimenting and testing hypotheses on how these transformations work

Question 20

How do mid-level representations seem to be optimised?

Answer 20

Mid-level representation appears to be optimised to allow subsequent transformation to support object recognition

Question 21

What is distributed encoding?

Answer 21

Object identity is not reflected in the activity of a single neuron but the pattern of activity in a larger population of neurons

Question 22

What are the advantages of distributed encoding?

Answer 22

• allows some cell death without representation failing (graceful degradation)
• allows new patterns to be stored without new cells, a fixed group of cells can store a variable number of objects
• it is consistent with measured cell properties -ie there are rarely all or nothing responses

Question 23

What is the disadvantage of distributed encoding?

Answer 23

It is harder for humans to understand

Question 24

What are face-selective neurons?

Answer 24

Found later in the ventral stream, there are cells that respond more strongly to specific faces regardless of the image used

Question 25

How do artifical DCNNs relate to face-selective neurons?

Answer 25

Artificial DCNNs produce similar results to the brain as later network layers closely resemble responses of face-selective cells

Question 26

What are Deepfakes?

Answer 26

Videos where a DCNN is used to map ones person’s face onto another. first show the network training videos of the target face, then the network maps the features of the face and their movements onto the source face. artificial DCNNs can convincingly manipulate facial identity

Question 27

What are object-selective areas? what type of objects are commonly studied

Answer 27

There are brain areas that respond to many classes of objects
often studied: faces, places, words and tools

Question 28

How does the brain process semantic content?

Answer 28

Responses in the brain show that processing semantic content produces similar results to processing visual objects which suggests similar processes are involved

Question 29

How does the brain change its process when given a task to identify humans?

Answer 29

Recording sites throughout the brain change their object selectivity and start responding more to humans. A face-selective area could become a car-selective area if given a car identification task

Question 30

How does the brain process visual content in dreams?

Answer 30

Similarly to content seen when awake. The responses in the brain are similar when dreaming all the way back to the early image representation in V1

Question 31

How do responses change when we focus on something? object vs spatial responses?

Answer 31

Responses are drawn towards attended content, more neurons will respond to the attended area of the image

• object responses are drawn towards attended object
• spatial responses drawn towards attended locations