User Research - Mobile Usability Testing - Bibliography

By Yaro Brock

Chandler, C., Lo, G., & Sinha, A. (2002). Multimodal Theater: Extending Low Fidelity Paper Prototyping to Multimodal Applications. Conference on Human Factors in Computing Systems (CHI), Student Posters (pp. 874-875). ACM Press.

This is a report of how a team approached low fidelity prototyping for multimodal applications. It promotes the idea that multimodal interfaces can be simulated using basic non computer based supplies such as paper, overhead projectors and sticky notes. They were able to successfully prototype such objects as a PDA, mp3 player and a mobile assistant for use in an automobile. This is a demonstration of how the difficulties that come along with multimodal development can be overcome.

Coutaz, J., Nigay, L., Salber, D., Blandford, A., May, J., & Young, R. (1995). Four Easy Pieces for Assessing the Usability of Multimodal Interaction: The Care Properties. IFIP International Conference on Human-Computer Interaction (pp. 115-120). London: Chapman & Hall.

This article explains how the authors have assessed multimodal systems. The focus is on understanding the user’s perception of the multimodal interface as well as describing the hardware’s status. The paper provides new terminology that can be used to describe both the user and the hardware. Some examples of topics covered are: Does the user know which modality is available? Do users use all modalities or get stuck using one? Do the users have the cognitive resources to use specific modalities? Do they choose the most efficient modality?

Jameson, A. (2002). Usability Issues and Methods for Mobile Multimodal Systems. ISCA Tutorial and Research Workshop on Multi-Modal Dialogue in Mobile Environments. Kloster Irsee, Germany.

This article explains issues that come up when dealing with multimodal interfaces. For instance, some modes require less cognitive resources than others, and yet making the choice of which mode to use requires cognitive resources. It also highlights how the environment and the multi-modal device is always clamoring for the user’s attention. It offers the suggestion that multi-modal devices need to be aware of context in order to best know how to present information and yet also notices that they are not capable of effectively doing this.

Kaikkonen, A., Kallio, T., Kekalainen, A., Kankainen, A., & Cankar, M. (2005). Usability Testing of Mobile Applications: A Comparison between Laboratory and Field Testing. Journal of Usability Studies , 1 (1), 4-46.

This article, as its name connotes, compares the results of usability tests in lab and field settings. It concludes that lab studies can be equally effective as field studies with less effort and cost. The authors make the point that usability inspections are not to find every detail, but to find the largest issues that can be rectified.

Kim, S., Kim, M., Kim, S., & Kang, H. (2004). Usability Test Equipment for Mobile Devices. DIS2004. Cambridge, Massachusets.

This article analyzes and compares different equipment that might be used to test mobile devices. The equipment types were “Environment-mounted”, “Device-mounted”, and “Subject-mounted.” The results of the study showed that the “Subject-mounted” devices showed the greatest promise for getting the best information from testing, but that there were many drawbacks to each method.

Kondratova, I., Lumsden, J., & Langton, N. (2006). Multimodal Field Data Entry: Performance and Usability issues. The Proceedings of the Joint International Conference on Computing and Decision Making in Civil and Building Engineering. Montreal, Canada.

This is a study looking at a multimodal PDA type device being used in an outdoor work environment. The article states that multimodal devices are especially important in the field because environmental factors make different input/output methods favorable at different times. For instance, extremely bright sunlight makes it difficult to read screens, therefore audio input might be preferable. Also, environmental sensors could actually send back information about a work sight that a person would not even have to input if sensors were added to the device. It also includes an interesting argument for lab studies that represent outdoor environments without actually doing field studies.

Lee, K., & Grice, R. (2004). Developing a New Usability Testing Method for Mobile Devices. Professional Communication Conference (pp. 115-217). Minneapolis: IEEE.

In this article, the authors introduce the audience to usability theory in relation to mobile devices then go on to describe methods for evaluating usability. They go into depth about heuristic methods and then describe a unique method of usability testing that they have created. This new method involves the utilization of heuristics as well as scenario and questionnaire based approaches. It is notable in that it takes seriously the mobile context of PDA's, cell phones and other such devices. It also takes into account the social aspects of their use and considers how social feedback will affect use.

Ryu, Y., & Smith, T. (2006). Reliability and Validity of the Mobile Phone Usability Questionaire (MPUQ). Journal of Usability Studies , 39-53.

This is an article that discusses the authors’ attempt to create a valid psychometric questionnaire. I found the discussion of different types of mobile user’s particularly interesting. They are described as, “Display Mavens”, “Mobile Elitists”, “Minimalists” and “Voice/Text Fanatics”. This article makes the point that different users have vastly different methods of using the device. It is very important that users are identified and the effects of the interface be considered in relation to all of these user profiles.

Zhang, D., & Adipat, B. (2005). Challenges, Methodologies, and Issues in the Usability Testing of Mobile Applications. International Journal of Human Computer Studies , 18, 293-308.

This article summarizes different techniques for doing usability tests and discusses how context, network reliability and bandwidth limitation affect usability. The article does a great job of laying out the challenges faced when dealing with mobile devices (context, connectivity, screen size, different resolutions, limited power and data entry modes). The authors emphasize the power that a tester has in a lab compared to in the field. This helped lead me to my conclusion that labs are still the superior place to do most usability studies.