Analysis of Think Aloud Protocols Generated During the Retrieval Process of an Online Database

학술정보데이터베이스 검색과정의 사고구술 프로토콜분석

  • 김종애 (경기대학교 문헌정보학과)
  • Received : 2011.08.28
  • Accepted : 2011.09.09
  • Published : 2011.09.30


This study aimed to improve our understanding of users' cognitive processes based on think aloud protocols generated during the retrieval process of an online database. It analyzed think aloud protocols generated by participants while performing pre-defined tasks using an online database. The protocols were encoded using the open coding method. As the results, four broad categories including retrieval, system format, system functionality, and information were drawn, and 21 detailed categories were generated. Based on the results, suggestions for the design of online databases were provided.


Think Aloud;Protocol Analysis;Online Database;Usability;User Study


  1. Alling, Emily and Rachael Naismith. 2007. "Protocol Analysis of a Federated Search Tool: Designing for Users." Internet Reference Services Quarterly, 12(1/2): 195-210.
  2. Anderson, J. R. 1976. Language, Memory, and Thought. Hillsdale, NJ: Erlbaum.
  3. Carroll, J. S. and J. W. Payne. 1977. Judgments about Crime and the Criminal: A Model and Method for Investigating Parole Decisions. In B. D. Scales (Ed.), Perspectives in law and psychology: The Criminal Justice System (Vol. 1). New York: Plenum.
  4. Ericsson, K. A. and H. Simon. 1984. Protocol Analysis: Verbal Reports as Data. Cambridge, MA: MIT Press.
  5. Ericsson, K. A. and H. Simon. 1993. Protocol Analysis. Cambridge, MA: Bradford Book.
  6. Gilhooly, K. and C. Green. 1996. Protocol Analysis: Theoretical Background. In Handbook of Qualitative Research Methods for Psychology and the Social Sciences, ed. Richardson, J. T. E., 43-74. Leicester: BPS Books.
  7. Hughes, J. and S. Parkes. 2003. "Trends in the Use of Verbal Protocol Analysis in Software Engineering Research." Behaviour & Information Technology, 22(2): 127-140.
  8. Isenberg, D. J. 1986. "Thinking and Managing: A Verbal Protocol Analysis of Managerial Problem Solving." Academy of Management Journal, 29: 775-788.
  9. Kelly, Diane, Nina Wacholder, Robert Rittman, Ying Sun, Paul Kantor, Sharon Small, and Tomek Strzalkowski. 2007. "Using Interview Data to Identify Evaluation Criteria for Interactive, Analytical Question- answering Systems." Journal of the American Society for Information Science and Technology, 58(7): 1032- 1043.
  10. Martin, Scott L. and R. J. Klimoski. 1990. "Use of Verbal Protocols to Trace Cognitions Associated with Self - and Supervisor Evaluations of Performance." Organizational Behavior and Human Decision Processes, 46: 135-154.
  11. Nisbett, R. E. and T. D. Wilson. 1977. "Telling More Than We Can Know: Verbal Reports on Mental Processes." Psychological Review, 84: 231-259.
  12. Novotny, Eric. 2004. "I Don't Think I Click: A Protocol Analysis Study of Use of a Library Online Catalog in the Internet Age." College & Research Libraries, 65(6): 525-537.
  13. Ortiz-Repiso, V. and P. Moscoso. 1999. "Webbased OPACs: Between Tradition and Innovation." Information Technology and Libraries, 18: 72-75.
  14. Payne, J. W., M. L. Braunstein, and J. S. Carroll. 1978. "Exploring Predecisional Behavior: An Alternative Approach to Decision Research." Organizational Behavior and Human Performance, 22: 17-44.
  15. Russo, J. E., E. J. Johnson, and D. L. Stephens. 1989. "The Validity of Verbal Protocols." Memory and Cognition, 17: 759- 769.
  16. Schweiger, D. M. 1983. "Is the Simultaneous Verbal Protocol a Viable Method for Studying Managerial Problem Solving and Decision Making?" Academy of Management Journal, 26: 185-192.
  17. Schweiger, D. M., C. R. Anderson, and E. A. Locke. 1985. "Complex Decision Making: A Longitudinal Study of Process and Performance." Organizational Behavior and Human Decision Processes, 36: 245-272.
  18. Simon, H. A. 1981. "Information-processing Models of Cognition." Journal of the American Society for Information Science, 364-377.
  19. Strauss, A. and J. Corbin. 1990. Basics of Qualitative Research: Grounded Theory Procedures and Techniques. CA: Sage Publications.
  20. Talja, S., R. Heinisuo, E. Kasesniemi, S. Kemppainen, S. Luukkainen, K. Pispa. et al. 1998. "Discourse Analysis of User Requests." Communications of the ACM, 41(4): 93-94.
  21. Weber, R. P. (1990). Basic Content Analysis (2nd ed.). Newbury Park, CA: Sage.
  22. Wu, R. C., M. S. Orr, M. Chignell, and S. E. Straus. 2008. "Usability of a Mobile Electronic Medical Record Prototype: A Verbal Protocol Analysis." Informatics for Health & Social Care, 33(2): 139- 149.