04 - Tracking Foundations und Optical Tracking Flashcards
Warum braucht man Tracking Systeme?
Die Idee der Tracking systeme ist es die Position und Orientierung eines reelen Objekts zu messen, um virtuelle Objekte besser mappen zu können
Für was wird Tracking in der AR verwendet?
- Augmentations anzeigen
- Interaktion
- Lokalisation vom User und Device
Welche Arten von Tracking gibt es in der AR?
- Tracken von Objekten
- Tracken von Features
- Tracken der Kameraposition
Was gehört in der AR noch zum Tracking?
Calibration und Registration
Welche Trackingsysteme werden nur wenig in AR, dafür vermehrt in VR verwendet?
- Mechanische
- Magnetische
- Akustische
Welche generellen Registrierungsprobleme gibt es?
In AR objects of real and virtual world have to be aligned with respect to each other
Humans are very sensible to visual errors
Registration issues in VR are harder to detect, basically one of the causes of cybersickness (visual-kinaesthetic and visual-proprioceptive conflict)
Visual capture decreases the issues related to registration errors in VR
Errors of a few pixels offset on the other hand are easily detectable in AR applications
Welche statischen Fehler gibt es bei der Registrierung?
Registration error if the viewpoint and the environment remain still
Causes
Optical distortion - if camera tracking is used
Errors in the tracking system - mechanical misalignments or offset of tracker and head
Incorrect viewing parameters - Field Of View (FOV) - Interpupillary distance (IPD) -> Abstand zwischen Augen, muss mitberücksichtigt werden
Welche dynamischen Fehler gibt es bei der Registrierung und was kann man dagegen tun?
Registration error during viewpoint and/or environment movement
Causes
Delays or lags
Overcoming dynamic errors
Reduce system lag - efficient algorithms and parallel computation
Reduce apparent lag - simplification of scenes
Match temporal streams - use of video information gathered from optical tracking
Predict future locations - help of accelerometers –prediction based on sensor speed and orientation (Wo wird sich User hinbewegen? Dann werden dafür vorberechnungen durchgeführt
Was ist Jitter?
Zittern bei der Latenz
Welche Jitter Fehler gibt es bei der Registrierung und was kann man dagegen tun?
Depending on used library even with static markers and static camera virtual objects could jump
Multiple reasons exist
Discretisation in the rasterisation process (optical tracking)
Jitter can be avoided through smoothing algorithms
Detection of larger jumps could point to an error
Potential solution
If 3rd measurement is in range of 1st measurement interpolation of 2nd could be performed by using 1st and 3rd measurement as base
Was ist Inertial Tracking und wie ist es aufgebaut?
Typically consists of 2 components
Gyroscopic compass
Accelerometer
Often used in combination with other tracking technology to provide additional information (e.g., optical, acoustic)
Was sind Vorteile von Interial Tracking?
No stationary tracking
Useful in large indoor areas where no other tracking is possible (e.g., buildings, caves)
- Kann verwendet werden wenn kein anderes Trackingsystem möglich ist
Was sind Nachteile von Intertial Tracking?
Accumulating position errors
Orientation stays precise if compasses are used
Wie funktioniert dabei das Intertiall Sensing?
Provides information about relative transformations of a target
Attempt to conserve either a given axis of rotation as in the case of a mechanical gyroscope or a position as in the case of an accelerometer
Mechanical gyroscope
A system based on the principle of conservation of the angular momentum
It states that an object rotated at high angular speed in the absence of external moments, conserves its angular momentum
If torque is exerted on a spinning mass, its axis of rotation will precess at right angles to both itself and the axis of exerted torque
Wie funktioniert das Intertial Sensing mit Accelerometer?
Measures the linear acceleration of an object to which it is attached to
Actually measures the force exerted on a mass since we cannot measure acceleration directly
Uses a known mass (proof mass) attached to a spring
Other end is attached to the accelerometers casing
If no force is applied, spring is in rest position
If force on the casing is applied, inertia causes mass to move and extend or compress spring
The displacement of the mass and extension/compression of the spring is proportional to the acceleration of the housing
Types of measurement sensors are potentiometric and piezoelectric
Wie funktioniert das Intertial Sensing mit Accelerometer?
Measures the linear acceleration of an object to which it is attached to
Actually measures the force exerted on a mass since we cannot measure acceleration directly
Uses a known mass (proof mass) attached to a spring
Other end is attached to the accelerometers casing
If no force is applied, spring is in rest position
If force on the casing is applied, inertia causes mass to move and extend or compress spring
The displacement of the mass and extension/compression of the spring is proportional to the acceleration of the housing
Types of measurement sensors are potentiometric and piezoelectric
Wie können beim Inertial Tracking Accelerometers und Gyroscpoes kombiniert werden?
Mechanical sensing
Placement on a gimbal locked surface
Was sind die Vorteile von kombiniertem Inertial Tracking?
Provides 6DOF acceleration and orientation measurement
Be always aware of the accumulative error
Measurements are always relative to last position
Can be a useful addition to other tracking technologies to increase the precision
Was ist GPS und wie wird es verwendet?
Originally military development
Now widely used in civil sector (e.g., navigation systems, precision farming, hiking, biking, geocaching)
Wie funktioniert GPS?
Satellites constantly send position and time values
Overlap of three spheres results in two points (one in space and one earth)
In theory reception of signals from three to four satellites required for trilateration and clock precision
Welche Frequenzen werden von GPS verwendet?
L1 – CA Coarse/acquisition
L2 – PY Precision/encrypted
Welche Alternativen gibt es zu GPS?
GLONASS (Russian), Galileo (Europe), MTSAT (Japan), Compass (China)
Welche Vorteile hat GPS?
Globale Tracking Area
Welche Nachteile hat GPS?
Himmel muss sichtbar sein
Verdeckungen von Häusern/Bäumen
Was ist optical Tracking?
Vast amount of possibilities
Use of different types of markers (e.g., active, passive)
Use of image processing to analyse the real world information (e.g. landmark recognition)
Inside Out vs. Outside In
Wie funktioniert Optical Tracking mit Infrarotlicht?
IR camera to track printed marker or IR sources
Commercial systems tend to be expensive, due to the required high quality camera equipment
No bright surroundings needed / desired
Collision with other IR light sources might occur (e.g., stereo emitters, sunlight)
Possibility to place IR sources in the surroundings, recording in that case might take place from a camera attached to a user
Welche 2 Varianten von Spatial Scan (klassische Verfahren) beim Optical Tracking gibt es?
Videometric
Beam scanning
Was ist Videometric Optical Tracking?
Example setup with sequentially fired LED patterns on the ceiling
Tracked with four cameras mounted on a users helmet
Was ist Beam scanning Optical Tracking?
Scanning optical beams on a reference
Sensors located on the target detect the time of sweep of the beams on their surface
Welche Verfahren werden heute hauptsächlich verwendet?
- Pattern recognition
- Matrix Code
Wie funktioniert Pattern recognition?
Only one camera is needed
Shape and size of the object to be recognised are known beforehand to the system
The recorded 2D pattern on the image is a function of the position and orientation of the target
Most common approach using pattern recognition is fiducial tracking implemented for example in ARToolKit or in Vuforia
Another early approach is the Matrix Code by Rekimoto
was ist Matrix Code?
Also known as CyberCode, first used in NaviCam project
Square shaped barcode that can identify and distinguish 216 possible markers
Matrix markers attached on real-world objects
Based on object and pattern recognition
Markers are attached to moving objects
Camera at a fixed position recording images of markers
Marker could be fixed and camera movable or both camera and markers are movable
Wie funktioniert Matrix Code?
After image is recorded (a) thresholding and binarisation are performed (b)
Connected components are identified (c) in order to detect and identify markers
Based on identified marker position, orientation, size and type the virtual object can be drawn on top or relative to the marker (d)
Welche Probleme gibt es bei Matrix Code?
Computationally intense
Line of sight to the camera
Bright surroundings might be necessary
Wie können zum Optical Tracking reflektive Marker verwendet werden?
Common concept to use reflective marker configuration
Reflect IR light emitted by IR LEDs
Marker configurations are recorded by cameras configured to only record IR light
Conflicts with other IR sources might occur (e.g. sync from shutter glasses, Kinect)
Was ist JanusVF?
Inside-Out tracking looks at projected fiducial patterns
Provides a closed loop between recognition and display of targets
Uses Single-Constraint-At-A-Time (SCAAT), Extended Kalman Filter (EKF), and ARToolKit Plus
Tracking data is provided to the system via VRPN
Restricted usage in 6 sided CAVEs
Very limited application field
Precise and cheap tracking method
Man projiziert Muster was man zum Tracken verwendet hinter sich in eine Cave und nimmt es mit einer Kamera auf
Welche Algorithmen verwenden Tracking Systeme ohne Marker?
SLAM
Was ist SLAM?
● Odometry is the use of data from the movement of actuators to estimate change in position over time (Man zeichnet Raum mit Landmarks (Kanten, Ecken) auf)
● Simultaneous Localisation And Mapping (SLAM)
Idea behind SLAM
Place a robot at an unknown location in an unknown environment
Have the robot build a map of the environment
Use the map to compute the robots location
Aus welchen Schritten besteht SLAM?
Landmark extraction Data association State estimation State update Landmark update
Wie funktioniert das Erkennen von SLAM Landmarks?
Objects that can be recognized over and over again
The objects should be re-observable, distinguishable from each other and stationary
Type of measurement can be based on different technology, e.g. spatial scan, TOF
Welche Algorithmen werden zur SLAM Landmark Extraction verwendet?
- Spike landmarks
- RANdom SAmpling Consensus (RANSAC)
Welche Probleme können beim SLAM Landmark Extraction auftreten?
Matching observed landmarks from different scans
Problems that might occur
Might not re-observe landmarks every time step
Might observe something as being a landmark but fail to ever see it again
Might wrongly associate a landmark to a previously seen landmark
Possibility to take close-by landmarks into account (e.g. checking Euclidian distance)
Was wird für die SLAM State Estimation verwendet?
- Kalman Filter
- Used to build a “stochastic map” of spatial relationships
Was passiert beim SLAM State Update?
Estimate we obtained for the robot position is not completely exact due to the odometry errors
Using the associated landmarks we can now calculate the displacement
Was passiert beim SLAM Landmark Update?
Update the state vector X and the covariance matrix P in the EKF
Purpose is to have more landmarks that can be matched, so the robot has more landmarks that can be matched
Was sind typische Anwendungsfelder von SLAM?
Robots (e.g. vacuum cleaners)
AR Tracking in general
Panoramic images
Welche Verfahren gibt es zum 3D Depth Sensing und wozu wird es verwendet?
- Stereo Cameras
- Traingulation
- Structured Light
- Time of Light
Um 3D Modelle zu erhalten
Wie funktioniert im 3D Depth Sensing Stereo Cameras?
Two displaced cameras observe different views of the scene
Depth is calculated based on relative positions of the object or feature perceived in the two cameras images
Wie funktioniert im 3D Depth Sensing Triangulation?
Laser projection of dot or line into the scene
Displacement of laser projector to the camera is known
Based on the perceived camera image location of the hit point can be calculated
Precise measurements possible
Wie funktioniert im 3D Depth Sensing Structured Light?
Fixed or variable pattern is projected in the scene
Projected pattern is recorded again by camera system
Distortion of the pattern can be perceived and used to determine the structure of the object it is projected on
Wie funktioniert im 3D Depth Sensing Time of Flight?
Measurement of the time from emitting light to the reflection of emitted light
Delay determines the distance
Calculation based on speed of light
Was ist Visual Inertial Odometry?
● Combination of SLAM-like approaches with inertial tracking
● Inertial tracking is in that case used to support the visual odometry
● Sensor fusion algorithms
● Current tending approach for tracking
● General problem with visual odometry is motion blur during fast movements
Wie können Szenen/Räume durch Tracking rekonstruiert werden?
● Current devices use IR time-of-flight tracking (HoloLens) or IR structured light (iPhone X) for reconstruction
● Only very rough depth maps available in real time
● Re-meshing required, points have to be interconnected
Wofür werden Raumrekonstruktionen verwendet?
Placement of augmentations in the scene
Tracking purposes
Generation of virtual objects
Advanced rendering techniques like occlusion handeling
