Lecture 10 Flashcards
What is the relationship between a world point M and the corresponding point m in the camera image?
A ray passing through a M and m will intersect at the camera centre.
What are the two main types of distortion? What can we do about it?
Radial distortion, could be barrel or pincushion distortion.
Tangential distortion, camera lenses not exactly parallel.
We can estimate the distortion and remove it.
What is registration?
The alignment of virtual objects with the real world.
What is the difference between Outside-In vs Inside-Out tracking?
Outside-In tracking has optical sensors which track landmarks attached to our object of interest, in Inside-Out the sensor is on the object of interest and the landmarks are around them.
What are the steps in marker-based tracking with the ARToolKit?
Take a video stream and search for markers, then find the marker 3D position and orientation, then identify the markers by matching them with a template, then using the position and orientation of the markers transform 3D virtual objects to align them with the markers, finally, render the 3D objects into the video frame.
How is template matching done with base ARToolKit? What are some problems?
Can be done by moving the window and searching for position with the most similarity, detecting the highest matching area and then comparing it with the template image by sliding it. This is done with a library of templates. Disadvantages are that the performance can be heavily affected by the number of templates in the library and it is sensitive to occlusion.
How is template matching done with ARTag?
The internal region of the marker is sampled with a 6x6 grid and a 0 or 1 digital symbol is assigned. This is useful because no pattern files need to be loaded and it is robust against occlusion.
How does PTAM mapping occur?
Uses five-point-pose algorithm with stereo vvision to compute pose(requires pair of frams, feature correspondances and a baseline), this creates the initial map.
New keyframes are created if there is a baseline to the other keyframes and tracking quality is good, Keypoints of keyframe will be saerched for in neighbouring keyframes and triangulated to create 3D points, new ones will be added to the map. The map is optimised using bundle adjustment to minimise reprojection error. Outlier measurement will be performed if the mapping thread is idle.
How is PTAM tracking done?
Tracking runs at 30Hz, In the first stage(coarse) the motion model is used to predict the current camera pose, Based on this all map points are projected into the current image, a fixed radius is searched around the projected position and a subset of found points are used to compute the camera pose.
In the second stage(Fine) the updated camera pose is used to project a larger subset of map points and searched for again to update the camera pose.
What are some of the limitations of PTAM?
Relocalisation has little invariance to viewpoint, there is no loop detection, it can only handle a small workspace, initialization also requires human intervention.
What is Panoramic mapping and tracking? What are the useful features for users?
involves mapping a panorama onto a cylinter, orientation tracking is done using key points.
It’s useful features are realtime performance on smart phones, drift-free orientation tracking, re-localization, and Loop-closing.
What is model-based tracking?
requires a 3D model of the scene and a prior estimate of the camera pose, the 3D model is projected into the camera’s view for every frame, computing the visible parts of edges, the direction of the edge normal is searched for the nearest edge in the video image.
How does VRLT work?
Visual reconstruction and localization toolkit works with real-time tracking in a wide area environment with a handheld device, it uses an offline reconstruction of pointcloud models from an omnidirectional video captured by a walker. Feature based localization is done by a remote server querying database.
In initialisation the client sends keyframes ot the server which it matches against triangulated features in the database, and then sends the pose back to the client. This is not fast enough for tracking, which uses a fast patch tracking method by projecting patches from the database of images according to the pose prior(works similar to PTAM).
