Blog #5: A Framework for Robust and Flexible Handling of Inputs with Uncertainty

Paper Title: A Framework for Robust and Flexible Handling of Inputs with Uncertainty


Authors: Julia Schwarz, Scott E. Hudson, Jennifer Mankoff, Andrew D. Wilson


Presentation Venue: UIST '10 Proceedings of the 23nd annual ACM symposium on User interface software and technology

Summary:

New input technologies like touch, recognition based input such as pen gestures and next generation interactions all provide for more natural user interfaces. However, these techniques all create inputs with some uncertainty. Conventional infrastructure lacks a method for easily handling uncertainty, and as a result input produced by these technologies is often converted to conventional events as quickly as possible, leading to a stunted interactive experience. The authors present a framework for handling input with uncertainty in a systematic, extensible, and easy to manipulate fashion.

Using the various touch inputs from the user, a probabilistic finite state machine can be developed and used to solve the ambiguity in certain inputs. These inputs include scenarios when the user touch the area half-way between the two options/buttons. In this case, probabilistic finite state machine can be used to resolve the conflicting inputs.

To illustrate this framework, they present several traditional interactors which have been extended to provide feedback about uncertain inputs and to allow for the possibility that in the end that input will be judged wrong or end up going to a different interactor. The six demonstrations conducted by the researchers included tiny buttons that were manipulable using touch input, a text box that could handle multiple interpretations of spoken input, a scrollbar that could respond to inexactly placed input, and buttons which were easier to click for people with motor impairments. The framework supports all of these interactions by carrying uncertainty forward all the way through selection of possible target interactors, interpretation by interactors, generation of uncertain candidate actions to take, and a mediation process that decides in a lazy fashion which actions should become final.

Discussion:

The paper talks about an excellent idea that's now being put to use. Today, more and more devices have touch screens. Soon, hardware buttons and controls will be replaced by touch screens on all the devices. When the primary method of interaction is touch, it becomes vital to resolve such annoying issues. This research has already been incorporated in many vitual keyboards. On iphones and iPads, when the user types words, he/she often makes mistakes since the letters are placed very close to each other. However, the algorithm computes the probabilities and displays the character that is most often times that character that the user intended to use.