Journal of Intelligence and Information Systems (지능정보연구)

Korea Intelligent Information System Society (한국지능정보시스템학회)
권호 표지
DOI QR코드

DOI QR Code

An Interactive Cooking Video Query Service System with Linked Data

링크드 데이터를 이용한 인터랙티브 요리 비디오 질의 서비스 시스템

  • Park, Woo-Ri (Department of Computer and Information Engineering, Inha University) ;
  • Oh, Kyeong-Jin (Department of Computer and Information Engineering, Inha University) ;
  • Hong, Myung-Duk (Department of Computer and Information Engineering, Inha University) ;
  • Jo, Geun-Sik (Department of Computer and Information Engineering, Inha University)
  • 박우리 (인하대학교 컴퓨터정보공학과) ;
  • 오경진 (인하대학교 컴퓨터정보공학과) ;
  • 홍명덕 (인하대학교 컴퓨터정보공학과) ;
  • 조근식 (인하대학교 컴퓨터정보공학과)
  • Received : 2014.05.30
  • Accepted : 2014.09.15
  • Published : 2014.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

The revolution of smart media such as smart phone, smart TV and tablets has brought easiness for people to get contents and related information anywhere and anytime. The characteristics of the smart media have changed user behavior for watching the contents from passive attitude into active one. Video is a kind of multimedia resources and widely used to provide information effectively. People not only watch video contents, but also search for related information to specific objects appeared in the contents. However, people have to use extra views or devices to find the information because the existing video contents provide no information through the contents. Therefore, the interaction between user and media is becoming a major concern. The demand for direct interaction and instant information is much increasing. Digital media environment is no longer expected to serve as a one-way information service, which requires user to search manually on the internet finding information they need. To solve the current inconvenience, an interactive service is needed to provide the information exchange function between people and video contents, or between people themselves. Recently, many researchers have recognized the importance of the requirements for interactive services, but only few services provide interactive video within restricted functionality. Only cooking domain is chosen for an interactive cooking video query service in this research. Cooking is receiving lots of people attention continuously. By using smart media devices, user can easily watch a cooking video. One-way information nature of cooking video does not allow to interactively getting more information about the certain contents, although due to the characteristics of videos, cooking videos provide various information such as cooking scenes and explanation for each recipe step. Cooking video indeed attracts academic researches to study and solve several problems related to cooking. However, just few studies focused on interactive services in cooking video and they still not sufficient to provide the interaction with users. In this paper, an interactive cooking video query service system with linked data to provide the interaction functionalities to users. A linked recipe schema is used to handle the linked data. The linked data approach is applied to construct queries in systematic manner when user interacts with cooking videos. We add some classes, data properties, and relations to the linked recipe schema because the current version of the schema is not enough to serve user interaction. A web crawler extracts recipe information from allrecipes.com. All extracted recipe information is transformed into ontology instances by using developed instance generator. To provide a query function, hundreds of questions in cooking video web sites such as BBC food, Foodista, Fine cooking are investigated and analyzed. After the analysis of the investigated questions, we summary the questions into four categories by question generalization. For the question generalization, the questions are clustered in eleven questions. The proposed system provides an environment associating UI (User Interface) and UX (User Experience) that allow user to watch cooking videos while obtaining the necessary additional information using extra information layer. User can use the proposed interactive cooking video system at both PC and mobile environments because responsive web design is applied for the proposed system. In addition, the proposed system enables the interaction between user and video in various smart media devices by employing linked data to provide information matching with the current context. Two methods are used to evaluate the proposed system. First, through a questionnaire-based method, computer system usability is measured by comparing the proposed system with the existing web site. Second, the answer accuracy for user interaction is measured to inspect to-be-offered information. The experimental results show that the proposed system receives a favorable evaluation and provides accurate answers for user interaction.

스마트 미디어 장치의 발달로 인하여 시공간적인 제약이 없이 비디오를 시청 가능한 환경이 제공됨에 따라 사용자의 시청행태가 수동적인 시청에서 능동적인 시청으로 계속해서 변화하고 있다. 사용자는 비디오를 시청하면서 비디오를 볼 뿐 아니라 관심 있는 내용에 대한 세부적인 정보를 검색한다. 그 결과 사용자와 미디어 장치간의 인터랙션이 주요 관심사로 등장하였다. 이러한 환경에서 사용자들은 일방적으로 정보를 제공해주는 것보다는 자신이 원하는 정보를 웹 검색을 통해 사용자 스스로 정보를 찾지 않고, 쉽고 빠르게 정보를 얻을 수 있는 방법의 필요성을 인식하게 되었으며 그에 따라 인터랙션을 직접 수행하는 것에 대한 요구가 증가하였다. 또한 많은 정보의 홍수 속에서 정확한 정보를 얻는 것이 중요한 이슈가 되었다. 이러한 사용자들의 요구사항을 만족시키기 위해 사용자 인터랙션 기능을 제공하고, 링크드 데이터를 적용한 시스템이 필요한 상황이다. 본 논문에서는 여러 분야 중에서 사람들이 가장 관심 있는 분야중 하나인 요리를 선택하여 문제점을 발견하고 개선하기 위한 방안을 살펴보았다. 요리는 사람들이 지속적인 관심을 갖는 분야이다. 레시피, 비디오, 텍스트와 같은 요리에 관련된 정보들이 끊임없이 증가하여 빅 데이터의 한 부분으로 발전하였지만 사용자와 요리 콘텐츠간의 인터랙션을 제공하는 방법과 기능이 부족하고, 정보가 부정확하다는 문제점을 가지고 있다. 사용자들은 쉽게 요리 비디오를 시청할 수 있지만 비디오는 단 방향으로만 정보를 제공하기 때문에 사용자들의 요구사항을 충족시키기 어렵고, 검색을 통해 정확한 정보를 얻는 것이 어렵다. 이러한 문제를 해결하기 위하여 본 논문에서는 요리 비디오 시청과 동시에 정보제공을 위한 UI(User Interface), UX(User Experience)를 통해 사용자의 편의성을 고려한 환경을 제시하고, 컨텍스트에 맞는 정확한 정보를 제공하기 위해 링크드 데이터를 이용하여 사용자와 비디오 간에 인터랙션을 위한 요리보조 서비스 시스템을 제안한다.

Keywords

References

  1. Baltrunas, L., B. Ludwig, S. Peer, and F. Ricci, "Context relevance assessment and exploitation in mobile recommender systems," Journal of Personal and Ubiquitous Computing, Vol.16, No.5(2012), 507-526. https://doi.org/10.1007/s00779-011-0417-x
  2. Batista, F., J. Paulo, N. Mamede, and P. Vaz, "Ontology construction: cooking domain," Technical Report, INESC-ID, Lisbon, 2006.
  3. Cairns, K., J. Johnston, and S. Baumann, "Caring About Food Doing Gender in the Foodie Kitchen," Gender & Society, Vol.24, No.5 (2010), 591-615. https://doi.org/10.1177/0891243210383419
  4. Cordier, A., E. Gaillard, and E. Nauer, "Man-Machine Collaboration to Acquire Cooking Adaptation Knowledge for the TAAABLE Case-Based Reasoning System," Proceedings of the 21st international conference companion on World Wide web, (2012), 1113-1120.
  5. Ha, I., K.-J. Oh, M.-D. Hong, Y.-H. Lee, A. N. Rosli, and G.-S. Jo, "Ontology-driven visualization system for semantic searching," Multimedia Tools and Applications, Vol.71, No.2(2011), 947-965.
  6. Hamada, R., I. Ide, and S. Sakai, "Associating cooking video with related textbook," Proceedings of the 2000 ACM workshops on Multimedia, (2000), 237-241.
  7. Hamada, R., J. Okabe, I. Ide, S. Satoh, S. Sakai, and H. Tanaka, "Cooking navi: Assistant for daily cooking in kitchen," Proceedings of the 13th annual ACM international conference on Multimedia, (2005), 371-374.
  8. Irizarry, J., M. Gheisari, G. Williams, and B. N. Walker, "InfoSPOT: A mobile Augmented Reality mehod for accessing building information through a situation awareness approach," Automation in Construction, Vol. 33, (2013), 11-23. https://doi.org/10.1016/j.autcon.2012.09.002
  9. Lewis, J. R., "IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use," International Journal of Human-Computer Interaction, Vol.7, No.1(1995), 57-78. https://doi.org/10.1080/10447319509526110
  10. Madathil, K. C., R. Koikkara, J. Obeid, J. S. Greenstein, I. C. Sanderson, K. Fryar, J. Moskowitz, and A. K. Gramopadhye, "An investigation of the efficacy of electronic consenting interfaces of research permissions management system in a hospital setting," International Journal of Medical Informatics, Vol.82, No.9(2013), 854-863. https://doi.org/10.1016/j.ijmedinf.2013.04.008
  11. Martins, F., J. P. Pardal, L. Franqueira, P. Arez, and N. J. Mamede, "Starting to cook a tutoring dialogue system," Proceedings of the Spoken Language Technology Workshop, (2008), 145-148.
  12. Ribeiro, R., F. Batista, J. P. Pardal, N. J. Manede, and H. S. Pinto, "Cooking an ontology," Proceedings of the 12th International Conference on Artificial Intelligence: Methodology, Systems, and Applications (AIMSA), Vol.4183, (2006), 213-221.
  13. Snae, C. and M. Bruckner, "FOODS: A Food-oriented Ontology-Driven System," Proceedings of the Second IEEE International Conference on Digital Ecosystems and Technologies, (2008), 168-176.
  14. Sonesson, U., B. Mattsson, T. Nybrant, and T. Ohlsson, "Industrial Processing versus Home Cooking: An Environmental Comparison between Three Ways to Prepare a Meal," A Journal of the Human Environment, Vol.34, No.4(2005), 414-421. https://doi.org/10.1579/0044-7447(2005)034[0414:IPVHCA]2.0.CO;2
  15. Schafer, U., F. Arnold, S. Ostermann, and S. Reifers, "Ingredients and Recipe for a Robust Mobile Speech-enabled Cooking Assistan for German," Proceedings of the 36th Annual German Conference on AI, Vol.8077, (2013), 212-223.
  16. Vildjiounaite, E., J. Kantorovitch, V. Kyllonen, I. Niskanen, M. Hillukkala, K. Virtanen, O. Vuorinen, S.-M. Makela, T. Keranen, J. Peltola, J. Mantyjarvi, and A. Tokmakoff, "Designing Socially Acceptable Multimodal Interaction in Cooking Assistants," Proceedings of 16th international conference on Intelligent user interfaces, (2011), 415-418.
  17. Yu, L., Q. Li, H. Xie, and Y. Cai, "Exploring Folksonomy and Cooking Procedures to Boost Cooking Recipe Recommendation," Proceedings of the 13th Asia-Pacific Web Conference on Web Technologies and Applications, Vol.6612, (2011), 119-130.