Chapter 6: 3D Perception Flashcards
- How do we understand the SPATIAL extent of the world?
What’s in front/behind?
How close/far is that predator/prey?
What shape is an object?
What is the size of something?
How do we go from
2-dimensional stimulation
to
3-dimensional experience?
??
The world is 3D and follows the rules of Euclidian geometry:
Parallel lines remain parallel as they are extended in space
Objects maintain the same size & shape as they move around in space.
Internal angles of a triangle always add up to 180 degrees, etc.
Euclidean geometry is just the fancy term for the geometry you learned in high school…
Notice that images projected onto the retina are NOT Euclidean!
Projective geometry
Investigates the mathematical relationships between objects in the environment and their optical projections on the retina or on a picture.
- Euclidean geometry of the 3-dimensional world turns into something quite different on the curved, 2-dimensional retina
Re-construct a Euclidean world from non-Euclidean stimulation.
The optical projections of objects are inherently ambiguous:
For example, all of the black lines shown below (straight, swigly, short, crooked) would produce exactly the same image on the observer’s retina. One of the great mysteries of perception is how the visual system is able to resolve this ambiguity to accurately perceive the 3D structure of the environment.
Shading information is inherently ambiguous up to a stretching or shearing transformation in depth.
It is not yet known how a single perceived surface is selected for the set of possible alternatives.
extreme accidental animated image from Web Activity 4.3, Object Ambiguity. See next slide for animation
even in 3D space, we can suffer from Accidental views of objects that lead us to the wrong interpretations of objects.
Depth cue
Information about the 3rd dimension (depth) of visual space.
Monocular depth cue
A depth cue or perceptual bias that is available even when the world is viewed with one eye alone.
Pictorial depth cue
A cue to distance or depth used by artists to depict 3-dimensional depth in 2-dimensional pictures. (Basically a monocular cue in a picture.)\
Binocular depth cue
A depth cue that relies on information from both eyes.
Primarily stereopsis in humans.
Stereopsis
From the Greek ‘stereo,’ meaning “solid”, and opsis, “appearance, sight”
A term that is most often used to refer to the perception of depth and 3-dimensional structure obtained on the basis of visual information deriving from two eyes by individuals with normally developed binocular vision.
Monocular Depth Cues / Perceptual Biases
Occlusion
Familiar Size
Resting on ground bias
Linear Perspective:
- Foreshortening
- Relative Size
- Relative Height
Shape from Texture
Ground plane Bias
Shape from Shading
Convexity Bias
Aerial Perspective
Depth of Field
Motion Parallax
Ocular-Motor Cues
Accommodation (Monocular)
Convergence & Divergence (Binocular)
Binocular Depth Cues Include…
Binocular Disparity
Stereopsis
Occlusion
A monocular cue to relative depth order in which, for example, one object obstructs the view of part of another object
Shapes overlapping each other
Familiar size
A monocular cue based on knowledge of the typical size of objects
Overriding familiar size is the basis of many “B” science fiction movies…
…
Godzilla
Three of the infinite number of scenes that could generate the retinal image in Figure 6.42
pennies
Resting on the ground bias
Unless there is information to the contrary, objects will be perceived as resting on the ground.
Information from shadows or indirect illumination can override the bias to perceive objects to be in contact with the ground.
Linear perspective
Monocular Cue to 3-Dimensional Space
Lines that are parallel in the 3-dimensional world will appear to converge in a 2-dimensional image as they extend into the distance.
This is a result of projective geometry.
Vanishing point
The apparent point at which parallel lines receding in depth converge.
The vanishing point below is on a horizon line.
Relative size & Foreshortening
Changes in shape due to linear perspective.
Foreshortening
Changes in surface orientation causes projected shape to change.
Monocular Bias
Relative Size:
Changes in surface depth causes projected size to change.
Monocular Bias
Closed, Open Door to Hallway picture.
Parallel lines in the image plane, such as those defining the door in (a), remain parallel in the image
Train tracks picture:
The two people lying across these train tracks are the same size in the image
Relative height
Below the horizon, objects higher in the visual field appear to be farther away.
The ratio of an object’s projected height relative to the height of the horizon specifies physical size in units of eye height.
Taller than eye height
vs.
Shorter than eye height
The height of an object’s base with respect to the horizon specifies its distance in depth.
This assumes that objects are in contact with the ground
Apparent distance can have a strong effect on apparent size.
The projected sizes of all 3 cylinders are the same, yet their apparent sizes are quite different.
The perception of 3D shape from texture (gradients)
OP Art (Optical Art)
In the 60s, an art style known as Optical Art emerged that made use of optical illusions.
Pieces by Bridget Riley that depict surfaces with contour textures.
As you can see, 2D images give rise to 3D perceptions.
The goal of Eric’s Master’s research was to model how the visual system interprets these images.
(Cartography, mechanical drawings, & computer graphics)
Eric Egan’s Study on Planar Cut Contours.
Imagine taking an object and slicing it up with a knife.
Those cut lines create planar cut contours.
Now lets take the simplest case where these cut lines are getting farther away from you.
When this is the case we can easily determine the relative distance between any 2 points by just counting the contours between them.
For example, this point is 4 contours from this point.
All we have to do is add a scaling parameter and we know the distance in depth z.
The problem with this simple model is that contour textures are rarely oriented to that we are looking directly at them.
We therefore improved our model to take into account the orientation of the planar cut contours.
It is very important that we take the orientation of the contours into account because it has a dramatic effect on our perceptions.
These 2 (black & white wavy blanket) images are of the same surface, the only thing that has changed is the orientation of the planar cut contours.
Psychophysics:
The Depth Profile Task
How to we measure people’s perceptions?
We use a depth profile task where we ask people to recreate the depth they see by moving these dots up and down.
Data with more complex object:
The red lines denote where we asked people to judge depth.
Down here, these black lines are show what the actual shape of the object is.
The red dots are people’s judgments.
The red lines are the model fits.
People’s judgments are most accurate when the planar cuts are not slanted.
When the planar cuts are slanted to the side, this effects peoples judgments of the horizontal lines.
When the planar cuts are slanted down, this effects peoples vertical judgments.
And the model is able to capture both of these distortions.
Ground Plane Bias
There is a strong bias to interpret scenes such that depth increases with height in the viewing plane. (Floor not ceiling)
Both images are identical, but one has been rotated 180 degrees..
There is no right-wall or left-wall bias so these images are bi-stable.
…
Shape from shading
The visual system can easily determine 3D shape despite variations in illumination and material (pictures of different pearls and dip-valleys under different lighting effects).
Convexity Bias
Convex interpretations are preferred over concave ones.
Concave regions can appear perceptually as convex, even when there is potential information from the cast shadow to specify the correct sign of relief.
Aerial perspective
A depth cue that is based on the implicit understanding that light is scattered by the atmosphere
Example: Haze
Purple mountains picture.
Depth of field
A monocular depth cue that is determined by the amount of accommodation required to focus an image.
When focusing on a distance object, our depth of field is large (or our plane of focus is thick).
When focusing on a close object, our depth of field is small (or our plane of focus is thin).
Special “tilt-shift” camera lens can be used to adjust depth of field to make large distant objects look like small close miniature models.
Non-accidental Properties
Properties of an image such as co-linearity, co-termination, or parallelism that seldom occur by accident within optical projections.
Thus, if lines in an image are parallel (or co-terminate), they will be interpreted perceptually as if they are parallel (or co-terminating) in the 3D environment.
(Also used to define Geons)
Accidental Alignments…
Are created when properties of an image occur by accident
Forced perspective
Forced perspective involves viewing a scene from an accidental viewpoint
a technique that employs optical illusion to make an object appear farther away, closer, larger or smaller than it actually is.
It is used primarily in photography, film-making, and architecture.
It manipulates human visual perception through the use of scaled objects and the correlation between them and the vantage point of the spectator or camera.
______ involves viewing a scene from an accidental viewpoint
Forced perspective
holding up the tower of pisa
Kissing the sphynx
These images both contain accidental co-terminations.
An Impossible Figure
This illusion is created by having an accidental co-termination of the object’s edges.
Weird, twisted block triangle picture.
This is a photograph of a real object, but its apparent shape is geometrically impossible.
In this version of the impossible triangle there is an accidental viewpoint of the sculpture so that the curves are (wrongly) interpreted as straight lines.
Continuous contours in an image are interpreted as continuous contours in the 3D environment.
From an accidental view, this interpretation may be illusory.
Familiar Size
Picture of woman “holding” cow in her hand.
In this case, familiar size is overridden by the accidental alignment of the hand and the cow
Ames Room
The trapezoidal shape of the room causes an accidental parallel alignment of the back wall when viewed through the peep hole.
Forced Perspective of Cinderella’s Castle
as it becomes taller, its proportions get smaller.
For example, using this method, the top spire of the castle is actually close to half of its apparent size.
Major elements of the castle were scaled and angled to give the illusion of distance and height, a method frequently used in Disney theme parks around the world.
Lord of the Rings Film-Making
Forced Perspective in action
They used a moving camera and shifting platforms.
Accidental properties in art:
anamorphosis
An anamorphosis is a distorted projection or perspective; especially an image distorted in such a way that it becomes visible only when viewed in a special manner.
“Ana - morphosis” are Greek words meaning “formed again.”
The chalk pictures where it looks like someone is stick a foot out into 3D
Hans Holbein painted the double portrait in with an odd object (slanted skul) at the feet of the two men.
Motion parallax
Images closer to the observer move faster across the visual field than images farther away.
The brain uses this information to calculate the distances of objects in the environment.
Trains example.
Pictures of Trees
This photo looks “flat” due to its lack of monocular cues, motion and stereo vision.
This photo “pops” with help
from other monocular cues
In real life both scenes would “pop” thanks to motion parallax and stereo.
Ocular-Motor Depth Cues
Accommodation: The process by which the eye changes its focus (in which the lens gets fatter as gaze is directed toward nearer objects)
Convergence: The ability of the two eyes to turn inward, often used to focus on nearer objects
Divergence: The ability of the two eyes to turn outward, often used to focus on farther objects
In principle, the distance of an object could be determined by the state of accommodation or convergence, but human observers are not very sensitive to this information
Accommodation
The process by which the eye changes its focus (in which the lens gets fatter as gaze is directed toward nearer objects)
Convergence
The ability of the two eyes to turn inward, often used to focus on nearer objects.
Divergence
The ability of the two eyes to turn outward, often used to focus on farther objects.
Binocular depth cues
Because…
The two 2D retinal images of a 3-dimensional world are not the same!
The holding one finger in front of the other one demo.
Binocular depth cues
A depth cue that relies on information from both eyes.
Primarily stereopsis in humans.
Binocular summation
The combination (or “summation”) of signals from each eye in ways that make performance on many tasks better with both eyes than with either eye alone.
Binocular disparity
The differences between the two retinal images of the same scene.
The basis for stereopsis.
Stereopsis
The ability to use binocular disparity as a depth cue.
A vivid perception of the 3-dimensionality of the world that is not available with monocular vision.
Visual fields vary, depending on the species
Rabbits and other prey animals generally have eyes on the sides of their heads.
They need to spot predators quickly from all directions.
Humans and predator animals tend to have eyes in the front of the face.
More binocular –> more accurate to spot prey.
Corresponding retinal points
A geometric concept
States that points on the retina of each eye where the monocular retinal images of a single object are formed are at the same distance from the FOVEA in each eye.
Crayons demo
This visual scene illustrates how geometric regularities are exploited by the visual system to achieve stereopsis from binocular disparity.
Horopter
Vieth-Muller circle
The location of objects whose images lie on the corresponding points.
The surface of zero disparity.
The Vieth–Müller circle and the horopter are technically different, but for our purposes you may consider them the same.
Horopter and Stereopsis
Objects on the horopter are seen as single images when viewed with both eyes.
Panum’s fusional area: The region of space, in front of and behind the horopter, within which binocular single vision is possible.
Objects closer or farther away from the horopter fall on non-corresponding points in the two eyes and are seen as two images
Diplopia: Double vision.
If visible in both eyes, stimuli falling outside of Panum’s fusional area will appear diplopic.
Amount of diplopia of an object determines the distance from the horopter.
Panum’s fusional area
The region of space, in front of and behind the horopter, within which binocular single vision is possible.
Diplopia
Double vision.
If visible in both eyes, stimuli falling outside of Panum’s fusional area will appear diplopic.
Amount of diplopia of an abject determines the distance from the horopter.
Horopters and Relative Disparity
The bigger the disparity, the farther away from the horopter of the object is.
Crossed disparity
The sign of disparity created by objects in front of the plane of the horopter
Images in front of the horopter are displaced to the left in the right eye and to the right in the left eye
Gaze turned towards the nose
Uncrossed disparity
The sign of disparity created by objects behind the plane of the horopter
Images behind the horopter are displaced to the right in the right eye and to the left in the left eye
Gaze turned outward, out from the nose,
How is stereopsis implemented in the human brain?
Input from two eyes must converge onto the same cell.
Many binocular neurons respond best when the retinal images are on corresponding points in the two retinas:
Neural basis for the horopter.
However, many other binocular neurons respond best when similar images occupy slightly different positions on the retinas of the two eyes (tuned to particular binocular disparity)
Stereoscope
A device for presenting one image to one eye and another image to the other eye.
Stereoscopes were a popular item in the 1900s
Many children in modern days had a ViewMaster, which is also a stereoscope.
3D movies and Binocular Vision
3D movies were popular in the 1950s and 60s and have made a resurgence in the late 80s and again in recent years.
For movies to appear 3D, each eye must receive a slightly different view of the scene (just like in real life).
Early methods for seeing movies in 3D involved “anaglyphic” glasses with a red lens on one eye and a blue lens on the other.
Current methods use polarized light and polarizing glasses to ensure that each eye sees a slightly different image among other techniques.
Anaglyph Stereograms
Early methods for seeing movies in 3D involved “anaglyphic” glasses with a red lens on one eye and a blue lens on the other.
Random Dot Stereogram (RDS)
A stereogram made of a large number of randomly placed dots
RDSs contain no monocular cues to depth
Stimuli visible stereoscopically in RDSs are cyclopean stimuli
Cyclopean: Referring to stimuli that are defined by binocular disparity alone
We live in a Cyclopean World.
Even though we have two eyes, we only perceive one world that is the combination of two.
*Weird dot pictrures.
Cyclopean
Referring to stimuli that are defined by binocular disparity alone
We live in a Cyclopean World.
Even though we have two eyes, we only perceive one world that is the combination of two.
Movie theaters now use polarized glasses to show movies in stereo
Active Shutter 3D Glasses for 3D TVs
require an infrared transmitter
Light waves…
involve oscillations in electric and magnetic fields.
Polarized lenses….
only pass light whose oscillations are oriented in a particular direction
Free fusion
The technique of converging (crossing) or diverging (uncrossing) the eyes in order to view a stereogram without a stereoscope
“Magic Eye” pictures rely on free fusion
Stereoblindness
An inability to make use of binocular disparity as a depth cue
About 5% of the population
Can result from a childhood visual disorder, such as strabismus, in which the two eyes are misaligned.
Most people who are stereoblind do not even realize it.
Binocular rivalry
The competition between the two eyes for control of visual perception, which is evident when completely different stimuli are presented to the two eyes.
If blue vertical bars are shown to one eye while orange horizontal bars are shown to the other, the two stimuli will battle for dominance
