Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
An overview of 3D user interfaces (UIs), covering definitions, examples, display devices, input devices, and guidelines for their development. The authors discuss various types of 3D UIs, including those with 3D input devices, 2D input devices with direct mappings to 3D, and 2D input devices used to interact with 3D virtual worlds. The document also explores display devices such as head-mounted displays (HMDs), volumetric displays, and handheld mobile displays, and input devices like 3D mice, tracking devices, and brain-computer interfaces.
Typology: Slides
1 / 58
This page cannot be seen from the preview
Don't miss anything!
3D UIs 101 Doug Bowman Welcome, Introduction, & Roadmap 3D UIs 101 3D UIs 201 User Studies and 3D UIs Guidelines for Developing 3D UIs Video Games: 3D UIs for the Masses The Wii Remote and You 3D UI and the Physical Environment Beyond Visual: Shape, Haptics and Actuation in 3D UI Conclusion
LaViola | Kruijff | Bowman | Poupyrev | Stuerzlinger 13
! Goal of this lecture
! Summarize research on 3D UIs ! 3D UIs in the lab ! Overview of 3D User Interfaces: Theory and Practice
! … all in 45 minutes!?
The goal of this lecture is to provide a foundation for the rest of the course. It will provide a whirlwind overview of research on 3D UIs to date, using our book 3D User Interfaces: Theory and Practice as a guide. Given the limited time, we’ll just present a few highlights, so that those not familiar with 3D UIs can understand the topics and issues presented in the rest of the course.
LaViola | Kruijff | Bowman | Poupyrev | Stuerzlinger 15
! Examples of 3D UIs
3D physical input, 3D virtual context
3D physical input, 2D virtual context
2D physical input, 3D virtual context
And yes, the Wii too!
The definitions on the previous slide lead to three categories of user interfaces that we consider 3D UIs: 1.3D input devices are used to interact with a 3D virtual world 2.3D input devices are used to interact with a 2D virtual world 3.2D input devices are used to interact (directly) with a 3D virtual world
LaViola | Kruijff | Bowman | Poupyrev | Stuerzlinger 16
Display devices for 3D UIs
LaViola | Kruijff | Bowman | Poupyrev | Stuerzlinger 18
! Visual displays for 3D UIs
! Standard monitor (mono/stereo) ! Handheld mobile displays ! Head-mounted/head-referenced ! Projected (usually stereo) ! single-screen ! multiple, surrounding screens ! Large tiled displays ! Volumetric displays
We’ll summarize the pros and cons of a few of the more common and/or interesting visual displays for 3D UIs.
LaViola | Kruijff | Bowman | Poupyrev | Stuerzlinger 19
! 3D with a monitor
3D UIs on the desktop are easier to achieve now than ever before. There are commercially-available autostereoscopic displays, making 3D viewing without glasses feasible. Adding a head tracker produces so-called “fishtank VR,” and a handheld tracking device (such as the Wii Remote) allows 3D input as well.
LaViola | Kruijff | Bowman | Poupyrev | Stuerzlinger 21
! Head-mounted displays (HMDs)
One of the most common display devices used for 3D UI applications is the head mounted display (HMD). With a tracking device attached to the device, it produces a stereoscopic view that moves relative to the user’s head position and orientation. Although traditionally the user cannot naturally see the real world, cameras are sometimes mounted on the HMD which allows it to display both real world video and graphical objects. In addition, some HMDs offer see-through options. This type of technology is used in augmented reality systems.
Since each eye is presented with one screen, HMDs allow for good stereoscopic viewing. These two screens are very close to the user’s eyes (1 to 2 inches). As a result, all viewable objects are behind the screen so any object clipping will appear to the user as being outside his/her field of view. A big disadvantage of HMDs is that can get heavy very quickly and, unfortunately, the higher the HMD’s quality, the heavier it usually is. Although HMDs are still used in many VR labs and entertainment centers, researchers and practitioners are rapidly moving towards projection-based display devices especially when high-resolution graphics are required.
Recently a high-resolution and wide FOV HMD came onto the market (www.sensics.com). It remains to be seen whether this will cause some high-end applications to return to HMDs.
LaViola | Kruijff | Bowman | Poupyrev | Stuerzlinger 22
! Surround-screen displays
Surround-screen displays, such as the CAVE™ are also extremely popular. Instead of attaching the displays to the user, they place the displays in the world. Such displays are typically rear-projected, stereoscopic, and head tracked. They range from two-screen L-shaped configurations to semi- cylindrical displays to spherical displays.
LaViola | Kruijff | Bowman | Poupyrev | Stuerzlinger 24
! UCSB AlloSphere
Another fully-surrounding display is the AlloSphere at UCSB. It’s a 3-story high spherical display with a “bridge” running through the center. When it is completed, it will offer 360-degree surround with high-resolution audio and stereoscopic video.
LaViola | Kruijff | Bowman | Poupyrev | Stuerzlinger 25
! Large tiled displays
The cheapest way to get a large display with very high-resolution is to tile multiple panels together. Here, 24 LCDs (without their casings) are tiled to produce a large, curved “desktop” display with more than 46 million pixels. 3D applications can run on such displays with the help of a small cluster of PCs and software (e.g., Chromium) that distributes the graphics rendering to each machine.
LaViola | Kruijff | Bowman | Poupyrev | Stuerzlinger 27
! 3D auditory displays
! Technologies: ! Speaker-based ! Headphone-based ! Uses: ! Virtual objects emitting sound (localization) ! Sensory substitution (sonification)
There are a number of different ways in which a 3D auditory system can be set up. A simple setup is to use stereo head phones. However, this restricts usage to only one person at a time. Another setup is to place speakers in certain logistic areas around the environment. This setup allows for more than one user to take part in the experience but is somewhat more complicated to setup and write software for.
There are two different ways, localization and sonification, in which sound can be used as an output medium in virtual environment applications. In localization, the goal is to generate three dimensional sound. In sonification, the goal is to turn certain types of information into sounds.
LaViola | Kruijff | Bowman | Poupyrev | Stuerzlinger 28
! Haptic displays
! Exoskeleton ! Robot arms ! Phantom ! Tactile devices
Haptics represents a critical component in virtual environment interaction. Allowing a user to touch and feel in the virtual world in the same way that they do in the physical world is extremely powerful. Unfortunately, haptic and tactile output device research has not made rapid progress.
There are essentially four different methods in which haptic and tactile feedback is generated. The first method is ground-referenced feedback which creates a physical link between the user and ground with the feedback relative to a single contact point. An example is the Sensable Phantom. The second method is body-referenced feedback which places a device on some part of the user’s body. An example of a body-referenced haptic device is Virtual Technologies’ CyberGrasp which is shown in the top picture. The third method for generating feedback is tactile which uses some type of oscillatory or vibrating device to stimulate the user’s tactile sense. Finally, the last method of generating feedback is via dermal tactile which stimulates the user’s nerves in the fingertips.
References: www.sensable.com www.immersion.com
LaViola | Kruijff | Bowman | Poupyrev | Stuerzlinger 30
Input devices for 3D UIs
LaViola | Kruijff | Bowman | Poupyrev | Stuerzlinger 31
! Input device characteristics
! Degrees of Freedom (DOFs) & DOF composition (integral vs. separable) ! Type of electronics : Digital vs. analog ! Range of reported values : discrete/continuous/hybrid ! Data type of reported values : Boolean vs. integer vs. floating point ! User action required : active/passive/hybrid ! Method of providing information : “push” vs. “pull” ! Intended use : locator, valuator, choice, … ! Frame of reference : relative vs. absolute ! Properties sensed : position, motion, force, …
There are many different ways to characterize input devices to be used in 3D UIs, some of which are shown here. In the 3D UI community, researchers often focus on degrees of freedom. But other characteristics can also be important. For example, a typical position tracker provides absolute position information. Some inertial input devices, like the Gyration GyroMouse, which some have seen as a replacement for position trackers, provide relative position information. This difference completely changes the way these devices are used in 3D interaction techniques.
