W9 - Spatial Frequency Analysis Flashcards
What does spatial frequency analysis seek to ask?
what are the building blocks used by the visual system to reconstruct the outside world?
What are the 2 hypotheses of potential building blocks of the visual system?
- 1st hypothesis = the cells are processing bars and edges
- 2nd hypothesis = V1 simple cells collect spatial frequency components (sine wave variations in luminance in different spatial scales), then reconstruct sine waves in later visual processing stage, V2, V3, V4, V5
What’s a sinewave?
Sine wave variation = gradual changes in luminance
Graph shows the graphical output of how luminance changes, from light to grey to dark and repeats in a periodical order
Why do we chose sinewaves over polymers?
V1 cells appear to extract spatial frequency components
EVIDENCE - Why do we think that the visual system uses sine wave modulations in luminance?
- Its mathematically possible from Fourier’s theorem (every image can be represented/broken down by a series of sine waves (could also be represented in polynomials)
- Receptive-field profiles / tuning properties of V1 simple cells can be modelled as sensitive to different SF components at particular locations)
What are the 2 two potential uses of SFA?
- Description/model of what the visual system actually does
- It’s a way to determine what any observe should see when presented with any stimulus, using SFA on ANY ORGANISM would see when shown ANYTHING (after tests you can predict what someone will see after been presented with any stimulus, eg. vision impaired, animals, infants
What are the 2 implications for using SFA?
- Determining how different people/organisms might see, eg. infants, visually impaired, animals
- Can also account for particular/specific percepts appear
eg. Einstein-Marilyn Monroe illusion, Mona Lisa smile
What are the building blocks of SFA?
The building blocks of SFA are sinewave modulations in luminance
Luminance = brightness, sinewave modulation = brightness varies as a sine wave function, eg.
What 3 things do you need to define to specify sine waves?
- Contrast © = how big are the sine waves? / how much energy you have at the spatial frequency
The difference between peak and trough compared to a background, the higher the contrast, the bigger the waves, eg. black/white = big waves, grey = minimal tiny waves - - Spatial frequency (SF = 1/wavelength) = how many peak-to-peaks/cycles do we have in a certain distance?
Top of the wave to the next top of the wave = wavelength
One cycle of the sinewave = from one peak to another peak - Phase (position 0) = what point does the sinewave start, midpoint, peak or trough?
Increasing or decreasing? A full cycle =360 degrees in the sinewave of phase angle, for a single sine wave, changing the phase merely changes where the location of the sine wave
What happens when you change the phase by 180 degrees?
the peak will move to where the trough was and vise were, location of peaks and troughs will change
= can produce different looking images
When the wavelengths are further apart…
…there are less cycles per degree (cpd) in a given distance
Smaller SINEwavelengths tend to have =
more cycles per degree
How is spatial frequency the inverse of a wavelength? (1/wavelength)
because the shorter the wavelength, the higher the spatial frequency
the longer the wavelength, the lower the spatial frequency
How do we quantify spatial frequency (SF) sine waves?
visual angle
1. accounts for distance
2. asks how many cycles do you have in a degree of visual angle?
the same visual angle can have different cpd depending on the number of SFs in an angle
What is visual angle important for?
- Important for our ability to RESOLVE the object, need to factor in the SIZE of the object and how FAR AWAY the object is
What happens to the spatial frequency content as the object gets further away?
a higher spatial frequency / more wavelength cycles per degree
is produced upon the same visual angle
What does the Spatial-frequency Spectrum plot?
SF spectrum shows the energy in the stimulus (sinewave variation) in each spatial frequency
shows how much CONTRAST at different spatial frequencies
Imagine 2 Spatial-frequency Spectrums, (SFSs) with different heights, what would this mean?
The left figure A has HIGHER CONTRAST (BIGGER SINE WAVES, peaks/troughs are further away from the midpoint) and lower spatial frequency compared to figure B
What happens to the SF spectra in going from a) to f)?
SFS spectra decreasing from A-F, turning to grey
- TO THE RIGHT OF THE X-AXIS = HIGHER SPATIAL FREQUENCIES
- LOWER ON THE Y-ASIS = LOWER CONTRASTS
GRAPH SHOWS DECREASING CONTRASTS WITH INCREASING SPATIAL FREQUENCIES
- F (NO LINE SHOWN) would have no contrast as it is the same colour
What would happen to the SF spectra if the phase of the sine wave was changed?
Phase doesn’t change anything on spectra
How are square waves constructed?
an image constructed by adding different sine waves together (eg. Fourier’s theorem) -by summing the fundamental + odd harmonic components
What is phase used for?
Make an image by adding different sinewaves together by changing the relative phase of some sinewave components relative to others (phase of SFs)
Cosine waves have different location of phase compared to sinewaves
How are sine and square waves different in apppearence?
Square wave modulation of luminance comes in from a mean luminance background from dark to light to dark very sharply
while sine waves show gradual changes in luminance from light to dark in periodical order
Sine waves needed to make a squarewave include fundamental and odd harmonics:
`
What is a fundamental?
What are odd harmonics?
Fundamental (1st sine wave) = start off with a sine wave with the same spatial scale as the square wave same spatial scale, has the same distance between peaks to peaks
Odd Harmonics = are integer multiples of the fundamental = with a higher spatial frequency compared to the fundamental such that the 3rd harmonic has a spatial frequency 3 times greater than the fundamental sine wave, but the contrast is a third of the fundemental
What is unique about the fundamental in a square wave?
The fundamental is the lowest spatial frequency sine wave in the image
What would first odd harmonic be for for a fundamental with 2 cycles per degree?
the 3rd harmonic would have 6 cycles per degree, as 2 * 3 = 6
What are the characteristics of the 5th harmonic? (assuming the fundamental only had 1 cycle per degree)
Then add the next odd number 5th harmonic, 5x spatial frequency of the fundamental, ⅕ of the wavelength in size, even lower contrast
What happens to the CONTRAST and the APPEARANCE of the squarewave you add more harmonics?
- The contrast DECREASES, eg. the 5th harmonic only has a 1/5 of the contrast of the fundamental
As you add more harmonics, the sinewave looks more sharper edged
At what contrast level would you be able to discriminate a square wave from a sine wave?
When the third harmonic is visible
What experiment would test the detectability of a sinewave from a squarewave?
How much contrast do we need to tell them apart?
- Use an experimental procedure - have 2 computer screens, one has the square wave and the other has a sinewave
- Start at 0% contrast = both grey screens
- Slowly increase the contrast, reach a point that the contrast of the sinewave becomes detectable at threshold
- At threshold, you can detect the fundamental, but all other contrasts are subthreshold/undetectable, and both sinewave and square wave look identical
- You can predict the contrast level at which you can discriminate a sinewave from a square wave when the third harmonic becomes visible - when the square wave looks different to the sine wave
Could you detect a squarewave from a sinewave at the precise contrast that a sinewave becomes detectable at threshold?
NO = At threshold, you can detect the fundamental, but all other contrasts are subthreshold/undetectable, and both sinewave and square wave look identical
How could you predict when a participant could discriminate between a sinewave and a squarewave?
- Work out CONTRAST DETECTION THRESHOLDS to predict when a participant can discriminate between the two waves when the harmonic component of the image becomes detectable
- IMPLICATIONS = You could PREDICT PERFORMANCE performance of the task by knowing the the components in the two images and the sensitivity of the participant’s contrast threshold
What is the difference between the square wave and the peaks add wave?
(Peaks add wave have little sharp light and dark bars in appearance around a grey background)
- The spectrum, the contrast and the spatial frequency doesn’t change… only the RELATIVE PHASE of the waves change
- The 3rd and 7th harmonics have a NEGATIVE NUMBER, so the phase angle has been shifted by 180 degrees
(Relative phase dramatically changes the image appearance and luminance)
What specific cells are tuned to specific spatial frequencies?
/
How are spatial frequencies processed at a cellular level?
V1 cells are tuned to particular oriented tuned columns at a particular orientation and cells within different spatial frequency components:
- Parvo cells are tuned to HIGH spatial frequencies, encode fine detail
- Magno cells are tuned to LOW spatial frequencies, encode course detail
What are the visual differences in transitions to light to dark between low and high SFs?
- LOW SFs have a gradual transition from light to dark and encode course information
- HIGH SFs have a rapid transition from light to dark and encode fine detail
What are the 2 methods used in SFA to predict visual performance?
Compare a generic lens system vs. visual system:
- MTF (Modulation Transfer Function), lens
- CSF (Contrast Sensitivity Function), human sensitivity
How is the MTF plotted?
the plot of the sensitivity of the system to different spatial frequencies (how does the system transfer/modify information that goes through it)
MTF = Percent transmission of a function of spatial frequency
Response of the system = stimulus spectrum * MTF
What 2 things is the MTF used for?
- Can break down a stimulus into its sinewave components
- Can predict the visibility of the stimulus based upon the visibility of sinewave components
How does SFs cycles per degree change percent transmission in a MTF?
- Any spatial frequencies with 0-10 cycles per degree (cpd) - percent transmission/100% of the contrast is transmitted
At a spatial frequency of 20 cycles of degree, only 50% of energy/contrast is transmitted
At a spatial frequency of 30 cycles of degree, sinewave contrast is reduced to 0%, any sinewave over 30 cycles per degree, none of energy gets through, contrast is reduced to 0%
If MTF is essentially the output image spectra, how does contrast change of the image at different cpd?
At 10 cpd there is 100% transmission - so the contrast of the sinewave is the same as the input image
At 20 cpd there is a 50% transmission - so the contrast of the sinewave will be a half of the input image
What happens if the lens gets degraded? (Fingerprints on the lens)
the MTF graph line changes (goes down quicker, closer to the left, takes less cpd to result in a loss of percent transmission
Imagine speakers but couldn’t listen to them before, but could see the MTF of 2 speakers:. The MTF would be percent transmission as a function of temporal frequency with sound waves
Blue = low pass filter, transmits low temporal frequencies
Red = high pass filter, transmits high temporal frequencies
All other things being equal, what do you do to determine speakers would you buy to listen to for Heavy metal?
- Break the music down into temporal frequency spectra (what sinewaves is the music composed off)
- heavy metal temporal frequency spectra has many low frequencies, eg. bass so better suited to headphones with lower temporal frequencies (blue)
What are Sound waves as a function of SFA?
sinewave variations of air density, (pressure waves)
How does hertz correlate to cpd?
Human auditory system goes from 20 herz-20 kilohertz (on the x-axis)
One hertz = one cycle per second, ie. temporary frequency = sinewave variations in air pressure per second
All other things being equal, what do you do to determine speakers would you buy to listen to for String quartets?
- Break the music down into temporal frequency spectra (what sinewaves is the music composed off)
- strings have higher delicate string work vibration will have higher temporal frequencies)
How would strings sound at lower temporal frequencies?
The strings would sound dampens/loss of detail on the lower temporal frequency headphones
How would heavy metal sound at higher temporal frequencies?
it would sound tinny
What is the human/VS equivalent of the MTF?
contrast sensitivity function (CSF)
How do we plot CSF?
plot of 1/threshold for sinewaves at each spatial frequency, giving sensitivity to different sinewaves at different SFs “window of visibility”
How does CSF differ to MTF?
the CSF can’t measure the input/output, must equate to how sensitive we are to different sinewaves
What are the 3 steps to construct the human CSF?
- Get sinewave components and then determine the human contrast detection threshold for sinewaves at different spatial frequencies (multiply together)
- Plot the contrast threshold for different spatial frequencies
- Make the CSF the inverse of the threshold for different spatial frequencies, it is reversed as it is more intuitive to think HIGHER scores mean BETTER performance
How do we interpret the curve in the CSF?
Anything beneath the curve is visible, anything above the curve is invisible
Campbell & Robson = What 2 things are needed to calculate CSF?
- how much contrast do you need for different spatial frequencies
- how much contrast a person needs to see the sinewaves (threshold), then plot 1/threshold
Why do we plot sensitivity over participant’s threshold? / What causes the shape of the CSF?
it’s more intuitive that it is inverted, more intuitive to think of the bigger the number the better the performance
What causes the loss of sensitivity at LOW SFs?
Loss of sensitivity at LOW SFS occurs as the tuning properties of simple cells in V1 are NOT TUNED TO REALLY LOW SPATIAL FREQUENCIES (uniform luminance) as it would mean equal amount of excitation AND inhibition
(They are only sensitive to changes in luminance of light OR dark that activate periods of (excitation or inhibition)
Once you get to 0cpd, the visual system cannot detect changes from light to dark, and the luminance appears uniform with 0 contrast (grey)
What causes the loss of sensitivity at HIGH SFs?
Loss of sensitivity at HIGH spatial frequencies is due to the sampling density and size of the receptive fields of the visual system, which is limited by how many visual cells are present, how tightly packed they are and how many feed into retinal ganglion cells
The optics of the brain blurs how high spatial frequencies to avoid issues of aliasing from high SFs
How do we design psychophysical studies to tap magno cells and parvo cells?
- Produce an image of a number with different spatial frequencies by convolving the original image with a spatial frequency filter (grey pictures) - (not assessed, spatial filter put on every location of the image and combining images to create new image)
Shows the animal represented at different spatial frequencies - taps different cells
Infants can’t process high spatial frequencies, while older people also begin to lose the ability to make out finer spatial frequencies
How would you determine the CSF of an infant?
- Preferential Looking/eye tracking: Present with 2 screens one with mean luminance/grey screen and 2nd with sinewave/lines
- if they can detect the sinewaves, they will preferentially look at a stimulus over a blank screen
- plot the infant’s contrast sensitivity by comparing preferential looking at grey screen to stimulus with different sinewave SFs
Campbell & Robson = Given the differences in the CSF, what would an infant not see when looking at a visual scene?
cannot detect high SFs, less ability to detect fine detail in the visual scene
You can work out what a child SEES at different ages, but how would you generate an image that represents what an infant sees? (2)
- Need to determine the infant’s CSF for different sinewaves
- Break down the image for sinewave components (fourier transform)
How would you test the CSF of a goldfish?
= use with food as an alternative form of preferential looking, ie. preferential feeding
Campbell & Robson = What part of the CSF tells you about a person’s spatial acuity threshold? / the upper frequency cutoff?
the bottom right part of the CSF with the highest contrast with the highest spatial frequencies
Imagine a vision test where all the letters are the same size but they slowly decrease in contrast, what is this measuring?
contrast sensitivity / acuity, size is kept constant
Imagine a vision test where all the letters are the same contrast but they slowly decrease in size, what is this measuring?
spatial acuity, not contrast acuity
What is the difference between 20/20 and 20/60 vision?
20/20 vision = both you and average person can resolve it at 6m
20/60 vision = you can resolve something at 20ft (6m) while the average person can resolve it at 60ft (18m)
What is the upper frequency cutoff / maximum spatial frequency for humans, cats and eagles?
Why do eagles have very high SFs?
- Humans = 30cpd
- Cats = 20cpd
- Eagles = 140cpd
The high spatial frequency content of the eagle is more beneficial to detecting prey from high heights above ground, so evolved to resolve very high spatial frequencies at a distance
Consider a cat and a human viewing the Norwegian-blue parrot. Why do the differences in the CSF of the cats and humans make sense given the differences in our behaviours?
- The Cat has higher sensitivity to low CFs, so the cat might be better at detecting the actual presence of the bird (outline, shape) compared to a human.
- The human might be better at detecting finer details of the bird,, eg. the feather structure, lines of the head.
- The cat will be better at detecting the bird in especially in subthreshold/low light conditions, since a cat is more of a predator to birds than humans, so the presence of the bird is more important than the fine detail of the bird
What 3 things do you need to predict the visibility of a stimulus?
- Object’s Fourier (SF) spectrum
- The observer’s CSF (contrast sensitivity function)
- Then perceived image is given by = Stimulus Fourier spectrum * CSF
What does our high sensitivity to SFs imply from an evolutionary basis?
Humans high sensitivity to SFs means we have cells tuned to very small receptive fields (might have evolved from what environments we have evolved in, eg. fine motor activity or living in forests with a lot of visual detail)
