Lecture 4: Tracking & Output Flashcards
Describe Camera-based tacking systems
A number of different systems rely on cameras: Usually two or more cameras are used in combinations with special “targets” or LEDs that are recognized in the camera images.
We need to identify a point to track.
This is often done using lights: Either an LED or a highly reflective object that is illuminated by a light source like a camera flash.
We can identify the lights in the camera images. If our point to track is visible in images
from multiple cameras, we can accurately compute its position.
How do we get the rotation with Camera-based tracking systems?
To get the rotation of an object, we need multiple points. Arranging our points in a uniquely identifiable pattern allows us to compute a rotation.
This can be used for full body tracking as well:
Applying trackable points all over a human body allows us to track movement of body parts. That is how mocap for movies, etc. works!
How well does camera-based tracking work? Name several advantages.
Camera-based systems can be very precise. Quality and latency is highly dependant
on the setup (amount of cameras, quality of cameras, …).
These optical setups can provide very precise and low-latency tracking. High-end tracking
cameras can do several hundred frames per second!
Tracking reflective balls also has the advantage that the tracker does not require power or
any connection!
Also many camera-based systems can be easily expanded with additional cameras to
cover very large tracking areas.
Describe the Camera-based tracking system from Oculus Constellation
The original Oculus Rift HMD used a similar setup with 2-3 cameras and infrared LEDs hidden on the headset.
The LEDs on the headset and controllers allow the system to identify the position and rotation of headset and controllers.
Describe the Camera-based tracking system from PSVR
Tracking of the Playstation VR system works basically the same as Constellation, but uses visible lights!
Visible light means it works best in dim
environments!
Describe the Camera-based tracking system from Wii Remote
The camera is in the remote.
The Wii “sensor bar” does not sense anything! Is contains simply a couple of IR LEDs. The camera in the front of the Wii remote can sense the sensor bar to orientate. The IMUs do the rest.
Name examples for cameras that creates a depth images
Kinect depth sensor:
The Microsoft Kinect not only featured a VGA camera and a microphone, but also a depth sensor. The depth sensor projects a grid of IR points and measures the distance for each point.
Stereo Cameras:
Stereo cameras also create a depth image, but use 2 cameras close to each other. It basically works using binocular vision!
Both cameras being fixed in relation to each other gives us the data necessary to compute a depth value for each pixel by matching the 2 camera images.
What does SLAM stands for? Briefly describe it.
Simultaneous Localisation And Mapping:
SLAM is an approach commonly used in robotics and navigation. The goal of slam is to create a map of the surroundings (mapping) and locate the position of the device within them (localization).
SLAM is very common in many scenarios from robot vacuum cleaners at home to self driving cars.
Which sensor types do SLAM need? What use WMR Headsets?
SLAM does not need to use cameras or even optical sensors, but can use any sensor. Commonly available VR systems are usually camera-based, however.
Oculus switched from Constellation tracking to inside-out SLAM-based methods using infrared cameras.
Other companies, like the Windows Mixed Reality Headsets, use normal cameras recording in the visible spectrum of light (basically webcams).
How many cameras do I need for SLAM?
Commonly we see either 2 or 4 cameras for VR SLAM-based tracking. This also influences how well the controllers work!
More cameras are obviously better!
What about controllers in Camera-based systems – Inside-Out ?
The controllers usually do not have cameras in them. Instead they still rely on outside-in tracking and are equipped with LEDs that are either visible (Windows Mixed Reality, etc.) or infrared (Oculus).
The tracking cameras in the headset see the controllers and compute their position and rotation, supported by IMUs in the controllers.
Not ideal, since we are reduced to pure inertial tracking when the controllers leave the field of view of the cameras in the HMD.
Camera-based systems – Inside-Out
What are the advantages and disadvantages of SLAM-based tracking?
SLAM-based tracking has the major advantage that the HMD works without any external tracking setup and is very easy to set up and use.
There are some downsides, however:
* Precision and stability are not considered ideal
* Does not work as well in dark environments
* Controller tracking is limited by the cameras on the HMD
What is Lighthouse Tracking/SteamVR?
A lighthouse is a fixed, artificial landmark that is visible during the day as well as night, thanks to its (rotating) light.
By placing multiple “lighthouses” at fixed positions in an area, a device sensing their light can compute its position and rotation.
What is needed for a lighthouse tracking?
The lighthouse tracking works with a
number of basestations. Each has flashing LEDs and rotating IR lasers sweeping the room.
Describe the technology behind the basestation for a lighthouse tracking
A basestation is an active device that acts as a stationary beacon sending IR light:
* LEDs flash for synchronization with the HMD
* 2 rotating lasers sweep the room (horizontally + vertically)
Multiple base stations synchronise (optical, with an optional cable or by bluetooth).
How do we track with basestations for lighthouse tracking?
The HMD and controllers are covered with sensors, that can detect the IR light emitted by the base stations.
The HMD/controllers do the tracking. Basestations are just landmarks. They require power, but no connection to a PC.
How does lighthouse tracking work? How is the system able to compute the position of the tracked device?
- LED flash: The LEDs are used for synchronization. Before each sweep from the lasers, they emit a flash of light.
- Laser Sweep: As the IR lasers sweep the room, the photodiodes on the HMD and controllers detect when they are “hit”.
From the timing between the flash and the hits, the system is able to compute the position of the tracked device.
How well does Lighthouse Tracking work? What can cause issues?
SteamVR Tracking is one of the most accurate systems on the consumer market!
The setup of the basestations is not as convenient as inside-out tracking and it requires the HMD and controllers to be covered in photodiodes.
SteamVR tracking also works without external light sources (at night), although a lot of very bright light (even sunlight) has potential to cause issues!
Name several Output Devices
For our purposes, we limit MR devices to:
Visuals
* HMDs
* projectors & displays
Audio
* headphones or speakers
Haptics
* very limited – vibration etc.
Describe the difference between Single and Multi-Display. Is an HMD a multi-display system?
Single-Display:
There is one screen showing the entire image.
In an HMD, there may be one screen per eye.
Multi-Display:
The image is spanning over multiple
displays.
A display may be a screen, a projector, etc.
An HMD may have multiple displays, but they show separate images, so it is not multi-
display.
Describe a Multi-Display as Output Device in more detail.
Give specs about the Powerwall at the LRZ
The Centre for Virtual Reality and Visualisation at the LRZ is equipped with a 6m x 3.15m powerwall. 2 Sony 4k cinema projectors with polarisation filters enable VR applications to show 3D content.
The images are computed by 4 graphics servers. Each server computes the images for one quadrant of the screen, to improve performance.
Give specs from the LRZ Cave
HMDs use spatial separation. Describe what that mean.
Which two display technolgies are commonly found in HMDs? Please list advantages and disadvantages from the technologies.
Justify why LCD is not necessarily worse for HMDs
What type of lenses do modern HMDs use? What are advantages and disadvantages?
Fresnel lenses.
Advantage:
Fresnel lenses can offer a wide FOV while being light and small!
Disadvantage:
Possibility of glare and god rays
Which problem do the use of lenses cause? How can we fix the problem?
distortion.
In order to present a non-distorted image, we distort the computed image using an inverse distortion of the one introduced by the lens.
