Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
A new interaction design framework that views users and systems as coupled dynamic systems, emphasizing continuous actions and feedback, negotiation of intentions, and uncertainty. It also explores the use of multimodal feedback and liquid representation of interaction. The framework aims to improve usability and adapt to human behavior.
Typology: Lecture notes
1 / 45
This page cannot be seen from the preview
Don't miss anything!
Roderick Murray-Smith, John Williamson
Department of Computing Science,
University of Glasgow &
Hamilton Institute, NUI Maynooth
http://www.dcs.gla.ac.uk/~rod
http://www.dcs.gla.ac.uk/~rod/Videos.html Bayesian Research Kitchen, Grasmere, 7
th^
Sept 2008.
-^
environment
fluid, dynamic manner.
and flow of control, changing fluidly as context determines.
-^
process, where user and system work together tocommunicate intention.
sides.
The interface is a mechanism for controlling the flow of information froma system^ –
an interactive system has therefore to ascertain the intention of the userwith the minimal effort on the part of the user.
-^
The interaction is formulated as a continuous control process, wherethe system is constantly engaged in recursively updating a distribution(inference) over the potential intentions of a user while providingfeedback of the results back at a range of timescales, which users canthen compare with their goals.
-^
User and system attempt to negotiate a satisfactory interpretation of theuser’s intention.
-^
Wide variety of sensing and display technologies that can be used toconstruct the physical aspects of a human-computer interface.^ –
Rich sensors, from accelerometers, to smart clothing, to GPS units, topressure sensors etc, create the potential for whole new ways of interactingwith computational devices in a range of contexts.
-^
Each of these has different information capacities, noise properties, delays,frequency responses, and other modality-specific characteristics.
-^
Sensors will get cheaper, and new ones will create as yet unimaginedinteraction possibilities
-^
Building interfaces that make use of possibly
high-dimensional, noisy
,
intermittently available
senses to create usable communication media
is a challenge.
-^
We need general frameworks which are not tied to specific sensing ordisplay devices, but generalise to wider classes of devices.
The display is to provide the user with information needed toexercise control. i.e. predict consequences of controlalternatives, evaluate status and plan control actions, or betterunderstand consequences of recent actions.
-^
Basic feedback loops^ –
Visual, audio, vibrotactile display of states of phone, or of distantevents, people or systems.
-^
Modality scheduling^ –
Order of presentation of information in different feedback channels.
-^
Mobile context^ –
Disturbances, lower attention span, fragmentary/intermittentinteraction.
Classic example of The Royal Majesty
“precise” position
-^
Location-aware audio & hapticfeedback
-^
Use tilt and bearing to get rapidexploration^ –
Project forward, find likelylocations in the future.
-^
Map browsing; includeuncertainty about where we are^ –
Show all the possible places wemight be, given a map of thearea
-^
User can scan around andproject further into the future.
-^
Augmented reality content isinterpreted by models whichgenerate multimodal feedback
-^
Gas (MC) shows inferred beliefs, but is less focussed on action and control
-^
Solid point has no distribution, therefore limited feedback for user. Has clearcontrol only when using low-noise, directly mapped inputs.
-^
Liquid form is not a true distribution, but does relate to control, and is bettersuited for guiding the user’s attention.
-^
Potential for dynamic change of properties (moving from true distribution tonegotiated one?)
Start with Monte Carlo samplesAdd molecular dynamicsRender with isocontour tracing
Long range attractor
Short range replusion
Equilibrium of attraction and repulsion(with damping)
Particles exert forceon each other^ Gaussian on each sample
Render the isocontour
We focus on the problem of interaction with sensors producingcontinuously varying measurements.
-^
The interaction is a closed-loop control process and theultimate control variable is the distribution over actionablegoals.
-^
The purpose of the system is to perform recursive evidenceupdates to infer the new goal distribution, forming a trajectorythrough the space of distributions. The space in which thistrajectory lies is the goal space;
-^
For example, discrete selection:
p^1
...p
simplex in n
n -d space
-^
Inference (should) result in a smooth trajectory in this space
-^
Large steps in entropy are unnatural & error-prone
-^
Information rate determines smoothness
-^
Give feedback to user about progress through this space. Byavoiding discrete state changes as long as possible, the needfor after-the fact correction system such as
undo
can be
minimised.
Information and Smoothness
Constraints
If a point
x
in the goal space is considered,
( x
n^
pi
log
p 2 i
is the Entropy at that point. The communication rate of thesystem is given by
dH
( x
)/ dt
There is assumed to be a maximum potential communicationbit-rate
b
max
group is one such upper bound, for example; the sampling rateof a sensor is another.
-^
If the process is to be controlled by the interactor, however, thebandwidth of the feedback must also lie within the user’s ability,as otherwise the interaction will be unpredictably unstable.
-^
So
b
max
= min(
bmaxin
, b
maxout
b
max
enforces a smoothness
constraint on the goal space trajectories; since
dH
( x
)/ dt
b
max
Link between display and goal
spaces
Liquid cursor is 2-D as in existing pointing techniques
-^
Dynamic properties allow gestures to be recognised.
-^
Multiple hypotheses can be maintained until sufficient evidenceis provided to effect an action
-^
System and user share a model of the distribution over targets
