Blog #27: Sensing Cognitive Multitasking for a Brain-Based Adaptive User Interface

Paper Title: Sensing Cognitive Multitasking for a Brain-Based Adaptive User Interface

Authors: Erin Solovey, Francine Lalooses, Krysta Chauncey, Douglas Weaver, Margarita Parasi, Matthias Scheutz, Angelo Sassaroli, Sergio Fantini, Paul Schermerhorn, Audrey Girouard and Robert Jacob

Author Bios:

Erin Solovey: is a postdoctoral fellow in the Humans and Automation Lab (HAL) at MIT

Francine Lalooses: is a PhD candidate at Tufts University and has a Bachelor's and Master's degree from Boston University

Krysta Chauncey: is a post doctorate researcher at Tufts University

Douglas Weaver: has a doctorate degree from Tufts University

Margarita Parasi: is working on a Master's degree at Tufts University

Angelo Sasaroli: is a research assistant professor at Tufts University and has a PhD from the University of Electro-Communication

Sergio Fantini: is a professor at Tufts University in the Biomedical Engineering Department

Paul Schermerhorn: is a post doctorate researcher at Tufts University and has studied at Indiana University

Audrey Girouard: is an assistant professor at The Queen's University and has a PhD from Tufts University

Robert Jacob: is a professor at Tufts University

Presentation Venue: CHI '11 Proceedings of the 2011 annual conference on Human factors in computing systems that took place at New York (ACM)

Summary:
Hypothesis: Cognitive miltitasking is a common element in daily life, and the researchers' human-robot system can be useful in recognizing these multitasking tasks and assisting with their execution. If the authors can create a system to detect a user's "to-do list" and allow them to multitask on different things at once, then those tasks will be completed faster., the user will become "understood" by the system, and a new kind of technology will be effectively used.

How the hypothesis was tested: The first experiment was designed to highlight three conditions: delay, dual-task and branching. The participants interacted with a simulation of a robot on Mars, sorting rocks. Based on the pattern/order of rock classification, measure data related to each of the three conditions listed above.

The second experiment was used to determine whether they could distinguish specific variations of the branching task. Branching was divided into two categories: Random branching and predictive branching. Also, the experiment followed the same basic procedure as the first experiment. However, there were only two experimental conditions

Results: The preliminary study returned a recognition accuracy of 68%. To the authors, this was promising. In the second study with the robot and the rocks, any result where the participant achieved less than a score of 70% were discarded because it was seen as the task being done incorrectly. In the last study, there was no significant statistical difference found between the random and predictive branching. The authors were able to construct a proof-of-concept model because they were able to differentiate between the three types of tasking and incorporate machine learning to it.

Discussion:
Effectiveness: I found these guys very bright. The technology they use is super advanced and the methodologies were complex when dealing with the proof-of-concept system. I don't know how much effect this will have in the HCI field because it didn't seem to me that there were any new inventions in this paper. The authors definitely achieved their goals.

Blog #26: Embodiment in Brain-Computer Interaction

Paper Title: Embodiment in Brain-Computer Interaction

Authors: Kenton O'Hara, Abigail Sellen and Richard Harper

Author Bios:

Kenton O'Hara: Microsoft Researcher in Cambridge in the Department of Socio Digital Systems.

Abigail Sellen: is a Principal Researcher at Microsoft Research Cambridge in the United Kingdom and a co-manager of Socio-Digital Systems, an interdisciplinary group with a focus on the human perspective in computing.

Richard Harper: is a Principal Researcher at Microsoft Research in Cambridge and co-manages the Socio-Digital Systems group.

Presentation Venue: CHI '11 Proceedings of the 2011 annual conference on Human factors in computing systems that took place at New York (ACM)

Summary:
Hypothesis: The key focus of this kind of interaction technique is to offer new interaction modalities for people with disabilities such as those with motor control impairments. If BCI technology can be effectively used and harnessed, then it can be used outside of the realm of just video gaming and possibly with the combination of other technologies.

How the hypothesis was tested: 16 participants took part in the study, made up of four distinct groups of people who knew each other. For each group there was a "host" participant responsible for assembling the group of people with whom they wanted to play the game just as they would if playing any other game at home. A social group had to consist of at least two people. The groups consisted of different family members and friend relationships as determined by the group host. The groups were as follows:
Group 1: One female (early 30s) and one male, in a relationship and cohabiting
Group 2: Father playing with his three children
Group 3: Married couple, daughter, her two stepsisters, the boyfriend of one of the stepsisters. One of the stepsisters had learning difficulties.
Group 4: Mother, father, son and daughter

Results: Watching and interpreting the game was not always straightforward and unambiguous. The authors noticed that other players and spectators were not always able to understand the intentions and actions of others in relation to the game, particularly when there was no bodily manifestation of their intention. 

Discussion:

Effectiveness: In this paper, the authors examined the Brain-Computer interaction within the context of social game play in the home. By this, they offered an alternative and complementary perspective on this kind of interaction. I found this paper pretty cool. I would have fun if I were able to play a game like this. II noticed that some players had some difficulty in controlling the ball sometimes. I think that the authors definitely showed how the power of BCI technology is tangible and applicable to video gaming.

Blog #25: TwitInfo: Aggregating and Visualizing Microblogs for Event Exploration

Paper Title: TwitInfo: Aggregating and Visualizing Microblogs for Event Exploration

Authors: Adam Marcus, Michael S. Bernstein, Osama Badar, David Karger, Samuel Madden and Robert Miller

Author Bios:

Adam Marcus

Michael Bernstein: is a graduate student focusing on human-computer interaction at MIT in the CSAIL. His research is on crowd-powered interfaces: interactive systems that embed human knowledge and activity. 

Osama Badar: is currently a member of the CSAIL at MIT

David Karger: is a member of the CSAIL in the EECS department at MIT. He is interested in information retrieval and analysis of algorithms.

Samuel Madden: is currently an associate professor in the EECS department at MIT. His primary research is in database systems.

Robert Miller: is an associate professor in the EECS department at MIT and leads the User Interface Design Group. His research interests include web automation and customization, automated text editing, end-user programming, usable security and other issues in HCI

Presentation Venue: CHI '11 Proceedings of the 2011 annual conference on Human factors in computing systems that took place at New York (ACM)

Summary:
Hypothesis: Twitinfo can provide a useful tool for summarizing and searching twitter for information about events and trends.

How the hypothesis was tested: The authors asked 12 participants to use Twitinfo to research different aspects of a recent event. During this part of the process they gathered usability feedback and observed which interface objects were useful or ignored. The second part of the testing involved adding a time limit. Participants were given 5 minutes to research the event using Twitinfo and then 5 minutes to compose a report about their findings. At the end of the session, the participants were interviewed about their reactions to the Twitingo system.

Results: The authors found that participants were able to reconstruct reasonably detailed information about events even without prior knowledge of it. They found that when users were performing the freedom exploration they tended to explore the largest peak thoroughly and read tweets completely. They also drilled in on the map and followed links to related articles. Most of the tweets were only used to confirm event details rather than general the information. When the time constraint was introduced, the focus shifted to skimming peak labels for a broad sense of the event a chronology, and a few people honed in on only one or two links to outside news sources to minimize time spent searching through repeated information.

Discussion:
Effectiveness: I found this a great okay. Although the authors were successful in creating their product, it didn't seem as useful as they thought it would be. It might potentially be useful in a smaller group setting, perhaps if a high school student wanted to read some first-hand posts about an event that occured in school.

Blog #24: Gesture Avatar: A Technique for Operating Mobile User Interfaces Using Gestures

Paper Title: Gesture Avatar: A Technique for Operating Mobile User Interfaces Using Gestures

Authors: Hao Lu and Yang Li

Author Bios:

Hao Lu: is currently a Senior Research Scientist working for Google. He spent time at the University of Washington as a research associate in computer science and engineering. He holds a PhD in Computer Science from the Chinese Academy of Science.

Yang Li: is currently a Senior Research Scientist working for Google. He spent time at the University of Washington as a research associate in computer science and engineering. He holds a PhD in Computer Science from the Chinese Academy of Sciences. 

Presentation Venue: CHI '11 Proceedings of the 2011 annual conference on Human factors in computing systems that took place at New York (ACM)

Summary:
Hypothesis: Gesture Avatar can provide a useful, viable solution to the problem of imprecise finger input on touch-screen surfaces. The paper also presents three separate hypothesis regarding GA's relation to Shift: 1) "H1" Gesture Avatar will be slower than Shift on larger targets, but faster on small targets

2) "H2" Gesture Avatar will have fewer errors than Shift, 3) "H3" Mobile Situations such as walking will decrease the time performance and increase the error rate of Shiftm but have little influence on Gesture Avatar.

How the hypothesis was tested: Participants were asked to test both Shift and Gesture Avatar, with half of the group starting on one technology and the other half starting with the other. Participants were also asked to complete tasks both sitting and walking. To begin with, they were asked to select different targets of varying size, complexity, and ambiguity. Also, the performance time was measured between the lap of the start target and the selection of the final target. The researchers aimed to address both letter ambiguity and commonness by using 24 different letters and controlling the distance between the objects and the number of letters used.

Results: This paper presents and explores Gesture Avatar, an application designed to combat the issue of impression from finger-based touch screen technology. The authors developed their product on an Android phone and tested it against the current Shift technology to better understand where it was limited and where it excelled. Overall, their results matched their hypothesis and the product itself received a positive reception from test subjects.

Discussion:

Effectiveness: I think the researchers did a very good job of testing and presenting this project. I am glad to see that there is significant strides being taken in this area of precision as it is a prevalent issue among touch-screens today.

Blog #23: User-Defined Motion Gestures for Mobile Interaction

Paper Title: User-Defined Motion Gestures for Mobile Interaction

Authors: Jaime Ruiz, Yang Li and Edward Lank

Author Bios:

Jaime Ruiz: is a 5th year doctoral student in HCI Lab in Cheriton School of Computer Science at the University of Waterloo. Her advisor is Dr. Edward Lank.

Yang Li: is a Senior Research Scientist at Google. Before joining Google's research team, Yang was a Research Associate in Computer Science & Engineering at the University of Washington and helped find the DUB a cross-campus HCI community.

Presentation Venue: CHI '11 Proceedings of the 2011 annual conference on Human factors in computing systems that took place at New York (ACM)

Summary:
Hypothesis: Although modern smartphones contain sensors to detect three-dimensional motion, there is a need for better understanding of best practices in motion-gesture design.

How the hypothesis was tested: The authors conducted a "guessability study" on participants by having them perform tasks and asking them what gesture/motion allows for the optimal mapping. For the experiment, users were told to treat the smartphone as a "magic black brick" because the authors removed all recognition technology from it so the users wouldn't possibly be influenced by anything of that nature. They were told to create gestures for performing tasks from scratch. The participants were recorded via audio and video. Data was also collected from a software on the phone for a "what was the user trying to do?" perspective.The study was conducted over all users who have previous and relevant smartphone experience

Results: Users most commonly used gestures not related to smartphone gestures. For example, viewing the homescreen for the smartphone had a very popular gesture as a result involving shaking the phone. Generally, the results gathered from the authors has a general agreement among gestures as well as the reasoning for the gestures. The authors therefore were allowed the luxury from the video "out-loud" process data to understand the user's thought process when creating the gesture. Tasks that were considered to be "opposite" of each other had similar gesture motions but were performed in "the opposite" direction.

Discussion:
Effectiveness: The authors wanted to create a way to have simple and optimal motion gestures to interact with a smartphone. What better way to do it than to let a set of smartphone users "create" such a set? It was an excellent idea from the authors to follow this. The paper was a good example of progress in the mobile interaction arena. The authors did achieve their goal, but it would have been interesting to follow up studies to verify their results.

Blog #22: Mid-air Pan-and-Zoom on Wall-sized Displays

Paper Title: Mid-air Pan-and-Zoom on Wall-sized Displays


Authors: Mathieu Nancel, Julie Wagner, Emmanuel Pietriga, Olivier Chapuis and Wendy Mackay


Author Bios:
Mathieu Nancel: is currently a PhD student in HCI in the University of Paris - Sud XI under the supervision of Michel Beaudouin-Lafon and Emmanuel Pietriga.


Julie Wagner: is a PhD student in the insity lab in Paris, working on new tangible interfaces and new interaction paradigms at large public displays


Emmanuel Pietriga: is currently a full-time research scientist working for INRIA Saclay-lie-de-France. He is also the interim leader of INRIA team In Situ.


Olivier Chapuis: is a research scientist at LRI. He is also a member of and team co-head of the In Situ research team.


Wendy Mackay: is a Research Director with INRIA Saclay in France, though currently on sabbatical at Stanford University. She is in charge of the research group, In Situ.


Presentation Venue: CHI '11 Proceedings of the 2011 annual conference on Human factors in computing systems that took place at New York (ACM)


Summary:
Hypothesis: The main hypothesis of the paper is that there is a need for more research on complex tasks when dealing with high resolution wall-sized displays. The authors made seven smaller hypotheses about how people best interact with tools. The authors had 7 separate hypotheses about each particular area of their design, but I combined this to make one general hypothesis.
How the hypothesis was tested: The authors conducted a series of pilot tests, pursued research, and performed empirical studies to narrow down all possible gesture and input methods for this system down to 12. They took into account performance (cost and accuracy), fatigue over periods of use of the input, ease of use, and natural mapping. The authors conducted an experiment to evaluate each of the 12 factors they discussed using in their system to see which were optimal. THe authors had their ideas about which of the 12 were optimal to begin with, but they tested their ideas to see if this was the case.
Results: After the experiment, the authors found that the two-handed gesture tasks were performed faster than the one-handed gesture tasks, involving smaller muscle groups for input interactions improves performance (providing higher guidance further improves it), and linearly-performed tasks were generally performed faster than circular ones. Circular gestures were slower because it was more often that participants overshot their target with circular gestures than with linear ones. After receiving feedback from the participants, they found that what the users were saying was in agreeance with their results from the gathered data.


Discussion:
Effectiveness: This paper was pretty interesting in the terms of field of study which has a lot of real-world applications such as crisis management/response, large-scale construction, and security management areas. Their sutdies were accurate, appropriate and extensive enough to gather relevant and meaningful data for designing a suitable system for the display. They definitely achieved their goals.

Blog #21: Human Model Evaluation in Interactive Supervised Learning

Paper Title: Human Model Evaluation in Interactive Supervised Learning


Authors: Rebecca Fiebrink, Perry Cook and Daniel Trueman


Author Bios:Rebecca Fiebrink: joined Princeton as an assistant professor in Computer Science and affiliated faculty in Music. She recently ompletedhr PhD in Computer Science at Princeton, and she spend January through August 2011 as a postdoc at the University of Washington. She works at the intersection of human-computer interaction, applied machine learning and music composition and performance.


Perry Cook: He researches but does not teach at Princeton University in the department of Computer Science and Department of Music.


Dan Trueman: Professor in the department of music at Princeton University




Presentation Venue: CHI '11 Proceedings of the 2011 annual conference on Human factors in computing systems that took place at New York (ACM)


Summary:
Hypothesis: If the user can be allowed to iteratively update the current state of a working machine learning model, then the results and actions taken from that model will be improved (in terms of quality).
How the hypothesis was tested: The authors conducted three studies of people applying supervised learning to their work in computer music. Study "A" was a user-centered design process, study "B" was an observatory study in which students were using the Wekinator in an assignment focussed on supervised learning, and study "C" was a case study with a professional composer to build a gesture-recognition system.
Results: From the studies, the authors gathered results and analyzed all of the results. From study "A", the authors saw that the participants iteratively re-trained the models by editing the training dataset. For study "B", the students re-trained the algorithm an average of 4.1 times per task, and the professional from study "C" re-trained it an average of 3.7 times per task. Cross-validation was used an average of 1 and 1.8 times per task, respectively. Direct evaluation was also present in the evaluation metric of the system. This was used more frequently than cross-validation. Participants in "A" strictly ued this measure, while the students and the professional in studies "B" and "C" used direct evaluation on an average of 4.8 to 5.4 times per task, respectively. Using cross-validation and direct evaluation, users were able to receive feedback on how their actions affected the outcomes. The overall results were that users were able to fully understand and use the system effectively. The wekinator allowed users to create more expressive/intelligent/quality models than with other methods/techniques.


Discussion:
Effectiveness: This paper shares a really good idea to have a system where you can tell the machine what is "good" or "bad" before it gives the final result. Being able to re-train the algorithm is very effective. The cost-benefit for it seems reasonable, so its easy to see this idea becoming more widespread before long. The authors achieved their goals and proved that their hypothesis was true.
Faults: I did not really find any faults with the system.