Week 2: deep learning in biological neurons and networks

Question 1

What does the cell membrane of a neuron do?

Answer 1

The cell membrane of neurons are specialised for electrical computations

Question 2

What are ions?

Answer 2

Atoms with electrical charges

Question 3

How do ions pass through the cell membrane?

Answer 3

Call membranes do not allow ions through. Ions must move through the ion channel or a pump

Question 4

How is an imbalance of ions across the cell membrane achieved?

Answer 4

Sodium/potassium pumps establish an imbalance of ions across the membrane

Question 5

What is the concentration of sodium and potassium around the neuron cell?

Answer 5

• much more sodium (Na+) on the outside of the cell

- a little more potassium (K+) inside the sell

Question 6

How do ions move towards balance?

Answer 6

Opening ion channels allows ions to pass though towards balance. There are different ions for sodium and potassium ions

Question 7

What is the resting state of the cell membrane?

Answer 7

Voltage of -70mV

Question 8

Is the inside or outside of the cell more positively/negatively charged?

Answer 8

The inside of the cell is negatively charged compared to the outside

Question 9

What do neurotransmitters do to a post-synaptic neuron?

Answer 9

Neurotransmitters released from a pre-synaptic cell can excite or inhibit activity in the post-synaptic neuron

• glutamate excites activity by opening the ion channel, causing the cell membrane to depolarise
• gamma-amino butyrate acid (GABA) inhibits activity by causing chloride to enter the cell, making the cell hyperpolarised
• when a neurotransmitter binds, the ligand-gated ion channels open

Question 10

What are two ways that an ion channel can open?

Answer 10

1. When a neurotransmitter binds to the ion channel, the ion channel changes (ligand-gated channels) shape and opens
2. The ion channel opens due to a change in membrane voltage (voltage-gated channels)

Question 11

What causes voltage-gated ion channels to open?

Answer 11

If the membrane polarisation reaches a threshold (due to many excitatory inputs), the voltage-gated ion channels will open

Question 12

What happens when the cell membrane reaches a certain threshold?

Answer 12

Causes an extreme depolarisation of the cell and the voltage-gated ion channels will open. This causes the action potential to “fire” meaning the cell passes on an electrical signal in the form of neurotransmitters to other cells. Passes down the axon to the next layer of neurons

Question 13

What is the strength of the firing rate of the neuron?

Answer 13

The firing rate increases as the strength of the inputs increases, at some point there is a maximum firing rate

Question 14

What is the threshold in biological neurons?

Answer 14

Around -15mV

Question 15

What happens to the cells once the action potential membrane has passed?

Answer 15

The ion channels are closed by their outer segment

Question 16

How are neurotransmitters released?

Answer 16

The arriving action potential causes voltage-gated ion channels to open, calcium ions flow into the cell and cause proteins in the axon terminal to change shape, releasing the neurotransmitter

Question 17

What is Hebb’s postulate and what does it mean?

Answer 17

“cells that fire together, wire together”

-when neurons fire together over and over, the brain strengthens responses to these patterns and learns to trigger these neurons together. humans learn by an unsupervised process by recognising patterns of activity we have seen before

Question 18

What is long term potentiation?

Answer 18

The lasting enhancement of synaptic connection by co-activation of presynaptic and postsynaptic neurons

Question 19

What do the network layers of an artificial network represent?

Answer 19

Layers of neurons at different levels of synapses

Question 20

What does the filter of an artificial network represent?

Answer 20

The integration of excitatory post-synaptic potentials and inhibitory post-synaptic potentials across the dendrite tree

Question 21

What does the threshold of an artificial network represent?

Answer 21

The activation of voltage-gated sodium channels by above-threshold depolarisations by summation of EPSPs and IPSPs

Question 22

What four sensors does the eye have for light? where are they all located?

Answer 22

Three types of colour-sensitive cones sensors (red, green, blue light intensity) and one type of rod sensor (overall light intensity), called photoreceptors. In central vision we find only cones, further out mostly rods. density drops off with distance from central vision

Question 23

What is a cortical map representation?

Answer 23

Where the visual image is “mapped” onto the cortical surface

Question 24

What is important when the eyes map an image onto the cortex?

Answer 24

Neighbouring locations must be represented next to each other so that the spatial extent of a filter represents a continuous piece of image

Question 25

Why is it important that spatial relationships are maintained when mapping an image onto the cortex?

Answer 25

• allows analysis of spatially-restricted patterns

- the dendrites of the next layer will sample from this layer and they have a limited extent

Question 26

How does the brain sample an image?

Answer 26

It magnifies the representation of important parts of the space: central vision, hands and face

Question 27

Why does the brain magnify some parts of an image?

Answer 27

To balance detailed analysis with computational load

-and because we can change what we sample in detail by moving our hands and eyes

Question 28

How does an artificial network sample an image differently to the brain?

Answer 28

Artificial network processes the whole image in detail, a human decides where to point the camera

Question 29

What is the receptive field in the biological system?

Answer 29

The part of the input image (the spatial extent of the image) that produces a positive response in a biological neuron

Question 30

What is the suppressive surround?

Answer 30

The part of the image that suppresses a response in a biological neuron

Question 31

How is an image first transformed in the brain?

Answer 31

The brain converts the amount of light at each location to the amount of contrast. An all white image would produce zero response but a bright point in the middle would have a positive response in the middle surrounded by a negative response`

Question 32

What does the receptive field refer to in an artificial network?

Answer 32

The spatial extent of a convolutional filter in the input image or previous feature map. this is a programmed parameter

Question 33

What is a limitation of knowledge of the biological receptive field?

Answer 33

It is hard to determine the spatial spread of a cells input.

Question 34

How does the activation of cones tell us about the image’s colour?

Answer 34

The relative activation of different cones must be compared to determine the colour

Question 35

What do retinal ganglion cells do?

Answer 35

Acts as filters, breaks down the image into different components at different spatial frequencies, different spatial scales, redness, greenness, blueness, yellowness, lightness, darkness and many combinations of these attributes

Question 36

What are the properties of bistratified retinal ganglion cells?

Answer 36

No suppressive surround around the activating centre, but limited spatial spread

Question 37

What are the properties of midget ganglion cells?

Answer 37

Small receptive fields and small suppressive surrounds with different colour inputs to the receptive field and the surround

Question 38

How are retinal ganglion cells as filters different from artificial network filters?

Answer 38

As we go from central to peripheral vision, the spatial extent of these filters increases, so the filters are not the same over the whole visual field. there can also be multiple receptive field sizes at the same location

Question 39

Why are there different sized receptive fields in the biological system?

Answer 39

The retina can break down any image into different components at different spatial frequencies and spatial scales, and can use them to see different things but put them back together to recreate the original image

Question 40

What is normalisation in the biological system?

Answer 40

The inhibition of activity by average nearby activity

Question 41

What is pooling in the biological system?

Answer 41

There is no distinct problem of computational overload in the biological system. progressive layers have smaller representations and more feature maps, so fewer neurons are used to sample the previous layer

Question 42

How can filters sometimes end up with similar weights in the biological system?

Answer 42

If a filter structure develops at one part of the feature map to compute something useful, similar filter structures are likely to develop elsewhere

Question 43

How do weights develop in the biological system?

Answer 43

Weights at each synapse develop independently from other neurons. so different synapses have different weights

Question 44

What is the main difference between artificial and biological networks in terms of feature maps?

Answer 44

Biological: feature maps don’t represent the whole image with the same detail, strong bias towards sensitive/important parts. hands and eyes can move to sample relevant information

Question 45

What is the main difference between artificial and biological networks in terms of filters?

Answer 45

Biological: filters have a limited spatial extent but not a fixed extent. multiple scales analysed at the same brain area ‘layer’. image layout is maintained in later stages, allowing analysis of spatial relationships at many levels

Question 46

What is the main difference between artificial and biological networks in terms of filter weights?

Answer 46

Biological: filter weights are not shared across each layer, but similar filters are likely to emerge in many places.

Question 47

What is the main difference between artificial and biological networks in terms of threshold?

Answer 47

Biological: maximum response is always part of the threshold

Question 48

What is the main difference between artificial and biological networks in terms of normalisation?

Answer 48

Biological: activity is normalised by local mean activity rather than global mean activity

Question 49

What is the main difference between artificial and biological networks in terms of learning?

Answer 49

Biological: learning is generally unsupervised