Usability Evaluation for Smart Phone Augmented Reality Application User Interface

스마트폰 증강현실 애플리케이션 사용자 인터페이스의 사용성 평가 방안

  • Received : 2010.12.07
  • Accepted : 2011.01.05
  • Published : 2011.02.28


Augmented reality applications offering location based service on smart phone require attention to users and new usability evaluation methods. The aim of this paper is to introduce new usability principles for application development and usability evaluation, and conduct expert evaluation on the augmented reality applications. We created a set of usability principles by analyzing characteristic of augmented reality applications, and conducted expert evaluation on three augmented reality applications primarily used by smart phone users. In expert evaluation and filtering procedure, we identified six usability problems that three applications have in common and the improvement guideline for the usability problems. The results of expert evaluation was verified by user test with a prototype that was developed using the guidelines. It is expect that the results of this study can be presented usability principles of the augmented reality application and a method of usability evaluation in the middle of new technology development process.


Usability User Interface;Expert Evaluation;Usability Principles;Augmented Reality;Smart Phone Application


  1. Bahr, G. S. and Nelson, M. M., "Development of a multiple heuristics evaluation table(MHET) to support software development and usability analysis," Universal access in humancomputer interaction : coping with diversity, Springer, Berlin/Heidelberg, 2007.
  2. Chincholle, D., Goldstein, M., Nyberg, M., and Eriksson, M., "Lost or Found? A Usability Evaluation of a Mobile Navigation and Location-Based Service," Proceedings of the 4th International Symposium on Mobile Human-Computer Interaction, Vol. 18, No. 20, 2002, pp. 211-224.
  3. Cockton, G. and Woolrych, A., "Understanding inspection methods : Lessons from an assessment of heuristic evaluation," In A. Blandford and J. Vanderdonckt, (Eds.), People and Computers XV, Springer-Verlag, 2001, pp. 171-192.
  4. Dunser, A., Grasset, R., and Billinghurst, M., "A survey of evaluation techniques used in augmented reality studies," ACM SIGGRAPH ASIA 2008 courses, Vol. 10, No. 13, 2008, pp. 1-27.
  5. Dünser, A., Grassert, G., Seichter, H., and Billinghurst, M., "Applying HCI principles to AR systems design," In : MRUI 2007, 2nd International Workshop at the IEEE Virtual Reality 2007 Conference, Charlotte, North Carolina, USA, 2007.
  6. Dykstra, D. J., "A Comparison of Heuristic Evaluation and Usability Testing : The Efficacy of a Domain-Specific Heuristic Checklist," Ph.D. dissertation, Department of Industrial Engineering, Texas A and M University, 1993.
  7. Gabbard, J. L. and Swan II, J. E., "Usability Engineering for Augmented Reality : Employing User-Based Studies to Inform Design," IEEE Transactions on Visualization and Computer Graphics, Vol. 14 No. 3, 2008, pp. 513-525.
  8. Gong, J. and Tarasewich, P., "Guidelines for Handheld Mobile Device Interface Design," Proc. of Decision Sciences Institute Annual Meeting, 2004.
  9. Henrysson, A., "Bringing Augmented Reality to Mobile Phones," PhD thesis, Linkoping University, 2007.
  10. Kim, H., Kim, M., Choi, J., and Ji, Y. G., "A study of Usability Evaluation for Tangible User Interface," In proceeding of AE International 2008 : 2nd International Conference on Applied Ergonomics, Las Vegas, Nevada, USA, Vol. 14, No. 17, 2008.
  11. Law, E. L. and Hvannberg, E. T., "Analysis of strategies for improving and estimating the effectiveness of heuristic evaluation," Proceedings of the third Nordic conference on Human-computer interaction, Vol. 23, No. 27, 2004, pp. 241-250.
  12. Lewis, J. R., "Psychometric evaluation of an after-scenario questionnaire for computer usability studies : the ASQ". In : ACM SIGCHI Bulletin, Vol. 23, 1991, pp. 78-81.
  13. Livingston, M. A., "Evaluating Human Factors in Augmented Reality Systems," IEEE Computer Graphics and Applications, Vol. 25 No. 6, 2005, pp. 6-9.
  14. Nielsen, J., "Heuristics Evaluation," In Nielson, J. and Mark, R. L., (Eds.), Usability Inspection Methods, John Wiley and Sons, Vol. 25, No. 62, New York, 1994.
  15. Nilsson, S. and Johansson, B., "User experience and acceptance of a mixed reality system in a naturalistic settin-a case study," In Proceedings of the International Symposium on Mixed and Augmented Reality, ISMAR' 06, IEEE, 2006. pp. 247-248.
  16. Nilsson, S. and Johansson, B., "Fun and usable : augmented reality instructions in a hospital setting," In Proceedings of the 2007 Australasian Computer-Human Interaction Conference, OZCHI, Adelaide, Australia, 2007, pp. 123-130.
  17. Shneiderman, B. and Plaisant, C., "Designing the User Interface : Strategies for Effective Human-Computer Interaction (4th Edition)," Pearson Addison Wesley, 2004.
  18. Ullmer, B. and Ishii H., "Emerging frameworks for tangible user interfaces," IBM Systems Journal, Vol. 39 No. 3-4, 2000, pp. 915-931.

Cited by

  1. User Interface Design Platform based on Usage Log Analysis vol.21, pp.4, 2016,
  2. Development of Mobile Location Based Service App Using Augmented Reality vol.18, pp.6, 2014,
  3. Usability Evaluation Criteria of Software GUI on Weapon System vol.16, pp.12, 2015,
  4. Development and User Satisfaction of a Mobile Phone Application for Image-based Dietary Assessment vol.22, pp.6, 2017,