06 - Spatial Augmented Reality and Tangible User Interfaces Flashcards
Was ist Spatial Augmented Reality?
● Information is displayed on physical objects
Actual objects or real world surfaces are used as display area
“In Spatially Augmented Reality (SAR), the user’s physical environment is augmented with images that are integrated directly in the user’s environment, not simply in their visual field“
Reinhold Behringer, Gudrun Klinker, David Mizell
● User is separated from the display
Scales up to a large number of users and allows for co-located interaction
Provides passive haptics sensation
Can avoid encumbering the user
Was ist die generelle Idee von Spatial Augmented Reality?
Calibration of display configuration using a video camera
Rendering perfomed using texture mapping hardware
Pair-wise registration and blending of projection overlaps
Was sind Vorteile von Spatial Augmented Reality?
Display surfaces may not be flat walls
Projection axis does not have to be orthogonal to displays
Flexible projection configuration
Projector’s intrinsic or extrinsic parameters not needed
Was ist das Luminous Room Konzept?
Projector-camera system
Captures user interaction and displays the result
Welche Probleme bestehen bei projektorbasierten Projektionen?
- Consistent occlusion
Consistent illumination
Welche Probleme gibt es bei der Projektor und multi projektor Kalibrierung?
Geometric correction can be performed
Colour correction can be performed
Intensity correction can be performed
Display on arbitrary surfaces should be possible
Wie sieht die Evolution von spatially continuous workspaces aus?
a) individual tasks with performed with a portable computer
b) table used as extension
c) pre-installed displays (table and wall) serve as shared workspace
I/O from portable computer is used with other workspaces (InfoTable and InfoWall) as well
Wie funktioniert das verschieben von spatial virtuellen Objekten?
(a) picking of virtual object with mouse cursor
(b) when cursor reaches edge of display virtual object is migrated to other display
(c) virtual object can be dragged to next display
(d) virtual object can be dropped on a physical object
in an object database a link between virtual and real object is established
Was ist ein anchored Cursor in der spatial AR?
Link between data on portable computer and data on InfoWall or InfoTable indicates interaction for other participants
Was ist eine Object aura in der spatial AR?
Area around a physical object
Data can be attached to registered and recognised physical objects
To bind these objects the virtual object has to enter the aura of a physical object
Wie kann ein Augmented Surface aufgebaut sein?
Communication between devices via wireless LAN
Projection on InfoTable and InfoWall
Identification of physical objects and portable computers via optical markers and cameras
Aura can be displayed to provide feedback on registration
Additional ceiling mounted moving camera used to increase resolution (Desksat)
Smaller area to cover focus set on desk objects
Wie funktioniert Projection Mapping in der Spatial AR?
● Outside installations often used for advertisement or art projects
● Complex calibrations corrections with help of projectorcamera (ProCam) systems used
● Approach of projector – camera systems
Project structured light (e.g. Grey Code) on desired display surface
Record the projected pattern displayed on the surface
Compare original data with projected data and determine offsets in geometry colour and brightness
Based on this data generate a distortion pattern in order to pre-distort image to be projected in a way, that the distortion through the environment is corrected
Was ist die AR Sandbox?
Consists of Kinect, projector, sandbox
Generation of real-time topography data with the Kinect depth sensor
Depth images are captured at 30FPS
Delay of 1 second is introduced in order
to filter moving objects (e.g. hands),
reduce the noise of the Kinect and
to fill missing data in the depth stream
Heat maps are generated and flow simulation is integrated
was ist die Augmented Climbing Wall?
Developed at Aalto University Finland
Gamification of indoor wall climbing
Multiple game modes available
Use of depth camera for tracking (Kinect v2)
Problems accurate tracking of climbing poses
Kinect SW is focused on reconstruction of standing or walking persons
Initial findings
Interactive graphics are best located near the holds where the attentional focus of the climber is naturally located
Fast moving graphics can be easily missed
Was ist das Mono-on-Mono Projekt?
Room scale projection
Only usable for dyadic interaction
Use of three projectors
Makes use of projecting behind other users, while each user is facing the other users projection with the back
Two renderings per object, each based on perspective of each user
Registration of the room performed with Kinects
Dynamic distortion on the user is also measured with a Kinect and used for correct perspective display
Interaction also performed with Kinect
Room geometry can be used for interaction as well
Was sind Pico Projectors?
● Battery driven
● Low brightness 15-50 lumens
● Max. 1280x720 resolution
● Image size max. 2.5m diagonal
● Projection distance max. 2.5m
● Growing range of products
● Video transmission via Bluetooth or HDMI
● Prices range between $100 and $500
● Highly portable
Was ist SixthSense und WUW?
Wearable gestural interface that augments the physical world with digital information and lets us use natural hand gestures to interact with that information
Consists of
Camera, mobile computing device, mirror pocket projector
Interaction
Projection of interaction space at variable locations
Coloured markers are attached to the fingers
Based on gestures
E.g. framing for taking pictures
Icon drawing with the index finger can be recognised (e.g. drawing an @-symbol for checking e-mails)
Was sind Tangible User Interfaces?
● A prop or a Tangible User Interface (TUI) is a physical object representing an object in a virtual world, which might have physical controllers mounted on it
● Its physical properties often suggest the use
Shape, weight, texture, centre of gravity, solidity, provide haptic information
● Tangible user interfaces provide access to virtual information through intuitive physical manipulation
● Seamless input interface which increases realism by providing haptic properties
F20 Was ist das Besondere an der Interaktion bei Tangible User Interfaces?
● TUIs try to integrate user interfaces in the surrounding world, not only physical objects but also surfaces, and spaces act as tangible embodiments of digital information
● Broadening interaction bandwidth
allowing users to “grasp & manipulate” foreground bits by coupling bits with physical objects, and
enabling users to be aware of background bits at the periphery using ambient media in an augmented space
● Not restricted to solid object vison goes also to gases and liquids zB Wii (Plastikgolfschläger/Lenkrad)
Was ist der TUISTER?
Incorporation of displays on TUI
Used to browse and access hierarchical structures
1 rotational DOF
Designed for two hand usage
Consists of display part as well as turntable sleeve
Contains sensors determining its orientation
Based on the orientation the primary display is chosen that it can be clearly read by the user
Wie kann mit dem TUISTER durch Hierarchien gebrowst werden?
Turning the display component leads to switching through entries in same hierarchy levels
Turning the sleeve clockwise leads to selection of the entry shown on the display
Turning the sleeve counter clockwise leads to moving backwards one hierarchy level
Ideally the dominant hand controls the display part while the non- dominant hand controls the sleeve
Was ist der Toolstone bei Tangible User Interfaces?
● Can provide as well an abstract shape (be generic) but still use an immediate mapping for scene manipulation
● Allow for more flexible and intuitive interaction
● Toolstone
Cube equipped with sensors (initially optical tracking)
Can be placed on six sides
When placed it can point in 8 different directions
Motion can be detected and mapped on user interaction
Wie werden Tools bei Tangible User Interfaces verwendet?
● Physical tools that are overloaded with logical functions
● Visual feedback presents the logical mode by using an SAR projection on tool
Immediate causal interconnection between tool and functionality provided
● Operation modes supported by a tool are defined by the shape of the tool itself
Classical prop approach
Was ist direktes oder indirektes Mapping vom Interaktionsraum bei Tangible User Interfaces?
Direct or indirect mapping of interaction space
Idea of mapping physical interaction into the design space
Direct mapping of interaction space results in visualisation of data in similar dimension and similar location
Indirect mapping for data manipulation relative to visualisation
Touch sensitive cube
Physical axes
Virtual mid air
Wie funktioniert Master based Input bei Tangible User Interfaces?
Use of virtual buttons and tangible input devices created by physical marker configurations
Menu item selection by tilting target marker
Connection establishment by proximity
Component based programming approach for marker interaction
Virtual “black hole” for object deletion
Basic application shows interaction techniques for marker based interactions
Zählen Sie ein paar Projekte zum Thema Tangible User Interface auf
iOrb, MagnifyingLens, T4T, Keyers, Lightglove, KITTY, ChairIO, ShoeSolesSense, selbstgemachte
Was ist der iOrb bei Tangible User Interfaces?
Single button
Contains inertial tracking to measure the device orientation
Similar to TUIster idea to manipulate menus
Was ist die Magnifying Lens bei Tangible User Interfaces?
Combination of markers with traditional input device in order to create tangible interface
Provides 6DOF tracking and additional device input (e.g. trackball, buttons)
Reference marker used in background
Implementation of the magic lens metaphor
Different approaches of visualisation implemented with lens metaphor
Was ist der T4T bei Tangible User Interfaces?
Use of relative motion of tablet for interaction
Rotation and translation of tablet are mapped on the transformation of a virtual object
Use of tracked cup as a dial
Rotational input of a cup equipped with a marker is mapped on a virtual dial
Was ist Keyers bei Tangible User Interfaces?
● Standard keyboards are often required or desirable for data input and communication, but not really applicable to wearable computing
● Devices, which are used to send a set of signals by hand
● Range from uni-ambic to multi-ambic (depending on the amount of buttons)
● A keyboard without board
● Typically one-handed devices
● Often come with training software
Was ist der Lightglove bei Tangible User Interfaces?
Attached on the wrist
Illuminates fingertips
Measures reflection if finger is bend
If finger is stretched no reflection occurs
Was ist KITTY bei Tangible user Interfaces?
● KITTY (Keyboard Independent Touch TYping)
Data input, wearable computing
Keyboard, is mapped on finger positions
Thumbs represent a row on the keyboard
Finger-thumb contact generates keystroke
Was ist ChairIO bei Tangible User Interfaces?
4 DOF (3 translatory DOFs, 1 rotatory DOF)
Aeris- chair equipped with an additional tracking system
Tracking could be optical, magnetic, or use mice wheels
Uses VRPN as a library for input processing
Used mainly for navigation in 3D environments
Terrain following
Flying
Huge mapping possibilities
3D First Person Shooters (FPS)
2D games possible as well
Can be used as a mouse replacement for desktop input
Was ist der ShoeSolesSense bei Tangible User Interfaces?
Alternative hands free input device
Pressure input mapped into four discrete cursor directions
Acts also as a display
Tactile vibration feedback
Temperature feedback
In virtual reality - supports walking in place metaphor
In wearable computing - provides ambient information
Prototype build with Arduino based controller as shoe sole inlay
Different foot shapes of 18 participants have been measured and used as input for the design of the pressure measurements
Socially very acceptable since basically invisible
Industrial application of foot based interfaces could be machine contro
Welche Tools kann man für selbstgemachten Tangible User Interfaces verwenden?
● For self assembly two components are often useful
● Potentiometer (Poti)
Resistor with a sliding contact
Adjustable voltage divider
Cheap, easy to gather data
● Arduino Board (www.arduino.cc/)
Ideal for homebrew installations
Connection of self-constructed sensors with board
Post-processing of signals
Fully programmable
Output via serial interface
Cheap and easy to use
