Korean Journal of Community Nutrition (대한지역사회영양학회지)

The Korean Society of Community Nutrition (대한지역사회영양학회)
권호 표지
DOI QR코드

DOI QR Code

Development and Validation of a Questionnaire on the Feasibility of a Mobile Dietary Self-Monitoring Application

식습관 관리 애플리케이션의 적용 가능성에 대한 설문지 개발 및 타당성 연구

  • Lee, Heejin (Department of Food and Nutrition, College of Human Ecology, Seoul National University) ;
  • Ahn, Jeong Sun (Functional Food Research Division, Ministry of Food and Drug Safety) ;
  • Lee, Jung Eun (Department of Food and Nutrition, College of Human Ecology, Seoul National University)
  • 이희진 (서울대학교 생활과학대학 식품영양학과) ;
  • 안정선 (식품의약품안전처 영양기능연구과) ;
  • 이정은 (서울대학교 생활과학대학 식품영양학과)
  • Received : 2021.12.02
  • Accepted : 2022.03.24
  • Published : 2022.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

Objectives: This study aimed to develop and assess the content validity and internal consistency of a questionnaire on the feasibility of mobile dietary self-monitoring applications. Methods: We developed a feasibility questionnaire to assess the overall usage, convenience, usefulness, and satisfaction of mobile dietary applications. The initial draft of the questionnaire contained 17 items with yes/no, multiple-choice, and open-ended questions and 52 items on 5-point Likert scales. To validate the content, ten experts evaluated the relevance of the items for each subscale using a 5-point scale. We calculated the item-level content validity index (I-CVI) and scale-level content validity index (S-CVI). A total of 102 adults answered the questionnaires which reflected the experts' reviews. We conducted an exploratory factor analysis to determine the underlying structure of responses and categorized convenience, usefulness, and satisfaction. We also calculated Cronbach's alpha coefficient to examine the internal consistency of items in each subscale. Results: The S-CVI score of the items was 0.86, and we removed items with an I-CVI score of < 0.80. We combined, revised, or separated some remaining items and added one item as per the experts' comments. As a result, we included 16 items about overall usage and 42 sub-questions. Based on the responses of the 102 adults, we performed exploratory factor analysis using the principal axis method. We retained items with a factor loading of > 0.40, resulting in a final set of 35 questions (convenience: 15, usefulness: 12, satisfaction: 8 items). The Cronbach's alpha values of the three scales were 0.93, 0.91, and 0.91 for 1) usefulness, 2) convenience, and 3) satisfaction, respectively. Conclusions: We developed a feasibility questionnaire for mobile dietary self-monitoring applications and examined its content validity and internal consistency. Our questionnaire has the potential to measure the feasibility of mobile dietary self-monitoring applications.

Keywords

Acknowledgement

This research was supported by the MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2018-2014-1-00720) supervised by the IITP (Institute for Information & communications Technology Promotion).

References

  1. World Health Organization. mHealth: New horizons for health through mobile technologies: Second global survey on eHealth. Geneva: World Health Organization; 2011. p. 66-71.
  2. Becker S, Miron-Shatz T, Schumacher N, Krocza J, Diamantidis C, Albrecht UV. mHealth 2.0: Experiences, possibilities, and perspectives. JMIR Mhealth Uhealth 2014; 2(2): e24. https://doi.org/10.2196/mhealth.3328
  3. Korea Internet & Security Agency (KISA) and Ministry of Science and ICT. 2017 survey on the internet usage [Internet]. Korea Internet & Security Agency; 2018 [cited 2021 Jun 11]. Available from: https://www.kisa.or.kr.
  4. National Information Society Agency (NIA). Internet usage survey of Korea 2020 [Internet]. National Information Society Agency; 2021 [cited 2021 Apr 04]. Available from: https://www.nia.or.kr.
  5. Franco RZ, Fallaize R, Lovegrove JA, Hwang F. Popular nutrition-related mobile apps: A feature assessment. JMIR Mhealth Uhealth 2016; 4(3): e85. https://doi.org/10.2196/mhealth.5846
  6. Zhou L, Saptono A, Setiawan IMA, Parmanto B. Making self-management mobile health apps accessible to people with disabilities: Qualitative single-subject study. J Med Internet Res 2020; 22(1): e15060.
  7. Research 2 Guidance. mHealth economics 2017 - Current status and future trends in mobile health [Internet]. 2017 [cited 2020 Nov 06]. Available from: https://research2guidance.com/325000-mobile-health-apps-available-in-2017/.
  8. Statistia. Global digital health market size between 2015 and 2025, by major segment [Internet]. Statista; 2020 [cited 2020 Nov 06]. Available from: https://www.statista.com/statistics/387867/value-of-worldwide-digital-health-market-forecast-by-segment/.
  9. GBD 2017 Causes of Death Collaborators. Global, regional, and national age-sex-specific mortality for 282 causes of death in 195 countries and territories, 1980-2017: A systematic analysis for the Global Burden of Disease Study 2017. Lancet 2018; 392(10159): 1736-1788. https://doi.org/10.1016/S0140-6736(18)32203-7
  10. GBD 2017 Diet Collaborators. Health effects of dietary risks in 195 countries, 1990-2017: A systematic analysis for the Global Burden of Disease Study 2017. Lancet 2019; 393(10184): 1958-1972. https://doi.org/10.1016/S0140-6736(19)30041-8
  11. Payne HE, Lister C, West JH, Bernhardt JM. Behavioral functionality of mobile apps in health interventions: A systematic review of the literature. JMIR Mhealth Uhealth 2015; 3(1): e20. https://doi.org/10.2196/mhealth.3335
  12. Teasdale N, Elhussein A, Butcher F, Piernas C, Cowburn G, Hartmann-Boyce J et al. Systematic review and meta-analysis of remotely delivered interventions using self-monitoring or tailored feedback to change dietary behavior. Am J Clin Nutr 2018; 107(2): 247-256. https://doi.org/10.1093/ajcn/nqx048
  13. Burke LE, Wang J, Sevick MA. Self-monitoring in weight loss: A systematic review of the literature. J Am Diet Assoc 2011; 111(1): 92-102. https://doi.org/10.1016/j.jada.2010.10.008
  14. Lee JE, Song S, Ahn JS, Kim Y, Lee JE. Use of a mobile application for self-monitoring dietary intake: Feasibility test and an intervention study. Nutrients 2017; 9(7): 748. https://doi.org/10.3390/nu9070748
  15. Mummah SA, Mathur M, King AC, Gardner CD, Sutton S. Mobile technology for vegetable consumption: A randomized controlled pilot study in overweight adults. JMIR Mhealth Uhealth 2016; 4(2): 1-13.
  16. Ashman AM, Collins CE, Brown LJ, Rae KM, Rollo ME. Validation of a smartphone image-based dietary assessment method for pregnant women. Nutrients 2017; 9(1): 73. https://doi.org/10.3390/nu9010073
  17. Tsai CC, Lee G, Raab F, Norman GJ, Sohn T, Griswold WG et al. Usability and feasibility of PmEB: A mobile phone application for monitoring real time caloric balance. Mob Netw Appl 2007; 12(2-3): 173-184. https://doi.org/10.1007/s11036-007-0014-4
  18. Zang J, Song J, Wang Z, Yao C, Ma J, Huang C et al. Acceptability and feasibility of smartphone-assisted 24 h recalls in the Chinese population. Public Health Nutr 2015; 18(18): 3272-3277. https://doi.org/10.1017/S1368980015000907
  19. Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system usability scale. Int J Hum-Comput Interact 2008; 24(6): 574-594. https://doi.org/10.1080/10447310802205776
  20. Ryu YS, Smith-Jackson TL. Reliability and validity of the mobile phone usability questionnaire (MPUQ). J Usability Stud 2006; 2(1): 39-53.
  21. Yen PY, Wantland D, Bakken S. Development of a customizable health IT usability evaluation scale. Proceedings of AMIA Annual Symposium. 2010 Nov 13; Washington, DC: p. 917-921.
  22. DiFilippo KN, Huang W, Chapman-Novakofski KM. A new tool for Nutrition App Quality Evaluation (AQEL): Development, validation, and reliability testing. JMIR Mhealth Uhealth 2017; 5(10): e163. https://doi.org/10.2196/mhealth.7441
  23. Ferrara G, Kim J, Lin S, Hua J, Seto E. A focused review of smartphone diet-tracking apps: Usability, functionality, coherence with behavior change theory, and comparative validity of nutrient intake and energy estimates. JMIR Mhealth Uhealth 2019; 7(5): e9232. https://doi.org/10.2196/mhealth.9232
  24. Boushey CJ, Spoden M, Zhu FM, Delp EJ, Kerr DA. New mobile methods for dietary assessment: Review of image-assisted and image-based dietary assessment methods. Proc Nutr Soc 2017; 76(3): 283-294. https://doi.org/10.1017/S0029665116002913
  25. Chin JP, Diehl VA, Norman KL. Development of an instrument measuring user satisfaction of the human-computer interface. Proceedings of the SIGCHI conference on Human factors in computing systems; 1988 May 15; Washington, DC: p. 213-218.
  26. Fruhling A, Lee S. Assessing the reliability, validity and adaptability of PSSUQ. Proceedings of 11th Americas Conference on Information Systems; 2005 Aug 11; Omaha: p. 2394-2402.
  27. Kollmann A, Riedl M, Kastner P, Schreier G, Ludvik B. Feasibility of a mobile phone-based data service for functional insulin treatment of type 1 diabetes mellitus patients. J Med Internet Res 2007; 9(5): e36. https://doi.org/10.2196/jmir.9.5.e36
  28. Welch J, Dowell S, Johnson CS. Feasibility of using a personal digital assistant to self-monitor diet and fluid intake: A pilot study. Nephrol Nurs J 2007; 34(1): 43-49.
  29. Kang TS, Eom MD, Nam KS, Park SY. Customized diet information system. J Korea Inst Inf Commun Eng 2010; 14(2): 430-436. https://doi.org/10.6109/JKIICE.2010.14.2.430
  30. Arab L, Wesseling-Perry K, Jardack P, Henry J, Winter A. Eight self-administered 24-hour dietary recalls using the internet are feasible in African Americans and Whites: The energetics study. J Am Diet Assoc 2010; 110(6): 857-864. https://doi.org/10.1016/j.jada.2010.03.024
  31. Finstad K. The usability metric for user experience. Interact Comput 2010; 22(5): 323-327. https://doi.org/10.1016/j.intcom.2010.04.004
  32. Fromme EK, Kenworthy-Heinige T, Hribar M. Developing an easy-to-use tablet computer application for assessing patient-reported outcomes in patients with cancer. Support Care Cancer 2011; 19(6): 815-822. https://doi.org/10.1007/s00520-010-0905-y
  33. Tariman JD, Berry DL, Halpenny B, Wolpin S, Schepp K. Validation and testing of the acceptability E-scale for web-based patient-reported outcomes in cancer care. Appl Nurs Res 2011; 24(1): 53-58. https://doi.org/10.1016/j.apnr.2009.04.003
  34. Kirwan M, Duncan MJ, Vandelanotte C, Mummery WK. Using smartphone technology to monitor physical activity in the 10,000 Steps program: A matched case-control trial. J Med Internet Res 2012; 14(2): e55. https://doi.org/10.2196/jmir.1950
  35. Kelly L, Jenkinson C, Ziebland S. Measuring the effects of online health information for patients: Item generation for an e-health impact questionnaire. Patient Educ Couns 2013; 93(3): 433-438. https://doi.org/10.1016/j.pec.2013.03.012
  36. Spook JE, Paulussen T, Kok G, Van Empelen P. Monitoring dietary intake and physical activity electronically: Feasibility, usability, and ecological validity of a mobile-based Ecological Momentary Assessment tool. J Med Internet Res 2013; 15(9): e2617.
  37. Anderson C, Henner T, Burkey J. Tablet computers in support of rural and frontier clinical practice. Int J Med Inform 2013; 82(11): 1046-1058. https://doi.org/10.1016/j.ijmedinf.2013.08.006
  38. Shah N, Jonassaint J, De Castro L. Patients welcome the sickle cell disease mobile application to record symptoms via technology (SMART). Hemoglobin 2014; 38(2): 99-103. https://doi.org/10.3109/03630269.2014.880716
  39. O'Connor E, Farrow M, Hatherly C. Randomized comparison of mobile and web-tools to provide dementia risk reduction education: Use, engagement and participant satisfaction. JMIR Ment Health 2014; 1(1): e4. https://doi.org/10.2196/mental.3654
  40. Teri S, Acai A, Griffith D, Mahmoud Q, Ma DW, Newton G. Student use and pedagogical impact of a mobile learning application. Biochem Mol Biol Educ 2014; 42(2): 121-135. https://doi.org/10.1002/bmb.20771
  41. Kwon GM, Kim AR, Kim SI. The activation methods of mobile service through usability testing: Focus on comparison mobile application YES24 and KYOBO bookstores. J Digit Des 2014; 14(3): 391-399.
  42. Shanahan CW, Sorensen-Alawad A, Carney BL, Persand I, Cruz A, Botticelli M et al. The implementation of an integrated information system for substance use screening in general medical settings. Appl Clin Inform 2014; 5(4): 878-894. https://doi.org/10.4338/ACI-2014-03-RA-0025
  43. Bae J, Kim S. Usability evaluation for mobile commerce of large-scale retailer by the increase of single-person households: Focused on comparison between E-mart and Homeplus. J Digit Des 2014; 14(1): 201-210.
  44. Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M. Mobile app rating scale: A new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth 2015; 3(1): e3422.
  45. Diamantidis CJ, Ginsberg JS, Yoffe M, Lucas L, Prakash D, Aggarwal S et al. Remote usability testing and satisfaction with a mobile health medication inquiry system in CKD. Clin J Am Soc Nephrol 2015; 10(8): 1364-1370. https://doi.org/10.2215/CJN.12591214
  46. Hutchesson MJ, Rollo ME, Callister R, Collins CE. Self-monitoring of dietary intake by young women: Online food records completed on computer or smartphone are as accurate as paper-based food records but more acceptable. J Acad Nutr Diet 2015; 115(1): 87-94. https://doi.org/10.1016/j.jand.2014.07.036
  47. Pretlow RA, Stock CM, Allison S, Roeger L. Treatment of child/adolescent obesity using the addiction model: A smartphone app pilot study. Child Obes 2015; 11(3): 248-259. https://doi.org/10.1089/chi.2014.0124
  48. van Osch M, Rovekamp A, Bergman-Agteres SN, Wijsman LW, Ooms SJ, Mooijaart SP et al. User preferences and usability of iVitality: Optimizing an innovative online research platform for home-based health monitoring. Patient Prefer Adherence 2015; 9: 857-867. https://doi.org/10.2147/ppa.s82510
  49. Agboola S, Palacholla RS, Centi A, Kvedar J, Jethwani K. A multimodal mHealth intervention (FeatForward) to improve physical activity behavior in patients with high cardiometabolic risk factors: Rationale and protocol for a randomized controlled trial. JMIR Res Protoc 2016; 5(2): e84. https://doi.org/10.2196/resprot.5489
  50. Mercer K, Giangregorio L, Schneider E, Chilana P, Li M, Grindrod K. Acceptance of commercially available wearable activity trackers among adults aged over 50 and with chronic illness: A mixed-methods evaluation. JMIR Mhealth Uhealth 2016; 4(1): e7. https://doi.org/10.2196/mhealth.4225
  51. Wilson K, Atkinson KM, Westeinde J, Bell C, Marty K, Fergusson D et al. An evaluation of the feasibility and usability of a proof of concept mobile app for adverse event reporting post influenza vaccination. Hum Vaccin Immunother 2016; 12(7): 1738-1748.
  52. Rhyner D, Loher H, Dehais J, Anthimopoulos M, Shevchik S, Botwey RH et al. Carbohydrate estimation by a mobile phone-based system versus self-estimations of individuals with type 1 diabetes mellitus: A comparative study. J Med Internet Res 2016; 18(5): e5567.
  53. Rickard N, Arjmand HA, Bakker D, Seabrook E. Development of a mobile phone app to support self-monitoring of emotional well-being: A mental health digital innovation. JMIR Ment Health 2016; 3(4): e49. https://doi.org/10.2196/mental.6202
  54. Ribeiro N, Moreira L, Barros A, Almeida AM, Santos-Silva F. Guidelines for a cancer prevention smartphone application: A mixed-methods study. Int J Med Inform 2016; 94: 134-142. https://doi.org/10.1016/j.ijmedinf.2016.07.007
  55. Svensson A, Magnusson M, Larsson C. Overcoming barriers: Adolescents' experiences using a mobile phone dietary assessment app. JMIR Mhealth Uhealth 2016; 4(3): e92. https://doi.org/10.2196/mhealth.5700
  56. Ward R, Taha KM. Patient involvement as experts in the development and assessment of a smartphone app as a patient education tool for the management of thalassemia and iron overload syndromes. Hemoglobin 2016; 40(5): 323-329. https://doi.org/10.1080/03630269.2016.1217875
  57. Wang J, Lam RW, Ho K, Attridge M, Lashewicz BM, Patten SB et al. Preferred features of e-mental health programs for prevention of major depression in male workers: Results from a Canadian National Survey. J Med Internet Res 2016; 18(6): e132. https://doi.org/10.2196/jmir.5685
  58. McManus DD, Chong JW, Soni A, Saczynski JS, Esa N, Napolitano C et al. PULSE-SMART: Pulse-based arrhythmia discrimination using a novel smartphone application. J Cardiovasc Electrophysiol 2016; 27(1): 51-57. https://doi.org/10.1111/jce.12842
  59. Nishiguchi S, Ito H, Yamada M, Yoshitomi H, Furu M, Ito T et al. Self-assessment of rheumatoid arthritis disease activity using a smartphone application. Development and 3-month feasibility study. Methods Inf Med 2016; 55(1): 65-69. https://doi.org/10.3414/ME14-01-0106
  60. English LL, Dunsmuir D, Kumbakumba E, Ansermino JM, Larson CP, Lester R et al. The Paediatric Risk Assessment (PARA) mobile app to reduce postdischarge child mortality: Design, usability, and feasibility for health care workers in Uganda. JMIR Mhealth Uhealth 2016; 4(1): e5167.
  61. Peterson CM, Apolzan JW, Wright C, Martin CK. Video chat technology to remotely quantify dietary, supplement and medication adherence in clinical trials. Br J Nutr 2016; 116(9): 1646-1655. https://doi.org/10.1017/S0007114516003524
  62. Spittle AJ, Olsen J, Kwong A, Doyle LW, Marschik PB, Einspieler C et al. The baby moves prospective cohort study protocol: Using a smartphone application with the general movements assessment to predict neurodevelopmental outcomes at age 2 years for extremely preterm or extremely low birthweight infants. BMJ Open 2016; 6(10): e013446. https://doi.org/10.1136/bmjopen-2016-013446
  63. Lee JY, Kim SI. A study on development of travel application UI design for improving user experience. J Digit Converg 2017; 15(2): 369-374. https://doi.org/10.14400/JDC.2017.15.2.369
  64. Oh YB, Kim WK. A study on mobile game UI design element considering user experience: Focused on management simulation game. Korean Soc Des Cult 2017; 23(2): 413-428.
  65. Lee E, Han S, Jo SH. Consumer choice of on-demand mHealth app services: Context and contents values using structural equation modeling. Int J Med Inform 2017; 97: 229-238. https://doi.org/10.1016/j.ijmedinf.2016.10.016
  66. Chen J, Lieffers J, Bauman A, Hanning R, Allman-Farinelli M. Designing health apps to support dietetic professional practice and their patients: Qualitative results from an international survey. JMIR Mhealth Uhealth 2017; 5(3): e40. https://doi.org/10.2196/mhealth.6945
  67. Tay I, Garland S, Gorelik A, Wark JD. Development and testing of a mobile phone app for self-monitoring of calcium intake in young women. JMIR Mhealth Uhealth 2017; 5(3): e27. https://doi.org/10.2196/mhealth.5717
  68. Coppock D, Zambo D, Moyo D, Tanthuma G, Chapman J, Lo Re III V et al. Development and usability of a smartphone application for tracking antiretroviral medication refill data for human immunodeficiency virus. Methods Inf Med 2017; 56(5): 351-359. https://doi.org/10.3414/me17-01-0045
  69. Shellington EM, Felfeli T, Shigematsu R, Gill DP, Petrella RJ. HealtheBrain: An innovative smartphone application to improve cognitive function in older adults. Mhealth 2017; 3: 17. https://doi.org/10.21037/mhealth.2017.04.05
  70. Beiwinkel T, Hey S, Bock O, Rossler W. Supportive mental health self-monitoring among smartphone users with psychological distress: Protocol for a fully mobile randomized controlled trial. Front Public Health 2017; 5: 249. https://doi.org/10.3389/fpubh.2017.00249
  71. Hayashi A, Yamaguchi S, Waki K, Fujiu K, Hanafusa N, Nishi T et al. Testing the feasibility and usability of a novel smartphone-based self-management support system for dialysis patients: A pilot study. JMIR Res Protoc 2017; 6(4): e63. https://doi.org/10.2196/resprot.7105
  72. Kelly MM, Hoonakker PL, Dean SM. Using an inpatient portal to engage families in pediatric hospital care. J Am Med Inform Assoc 2017; 24(1): 153-161. https://doi.org/10.1093/jamia/ocw070
  73. Herrmann LK, Kim J. The fitness of apps: A theory-based examination of mobile fitness app usage over 5 months. Mhealth 2017; 3: 2. https://doi.org/10.21037/mhealth.2017.01.03
  74. Kim SW, Lee GY, Yu HY, Jung EI, Lee JY, Kim SY et al. Development and feasibility of smartphone application for cognitive-behavioural case management of individuals with early psychosis. Early Interv Psychiatry 2018; 12(6): 1087-1093. https://doi.org/10.1111/eip.12418
  75. Brooke J. SUS: A quick and dirty usability scale. London: Taylor and Francis; 1996.
  76. Fakih El Khoury C, Karavetian M, Halfens RJG, Crutzen R, Khoja L, Schols JMGA. The effects of dietary mobile apps on nutritional outcomes in adults with chronic diseases: A systematic review and meta-analysis. J Acad Nutr Diet 2019; 119(4): 626-651. https://doi.org/10.1016/j.jand.2018.11.010
  77. Dorsch MP, Cornellier ML, Poggi AD, Bilgen F, Chen P, Wu C et al. Effects of a novel contextual just-in-time mobile app intervention (lowsalt4life) on sodium intake in adults with hypertension: Pilot randomized controlled trial. JMIR Mhealth Uhealth 2020; 8(8): e16696. https://doi.org/10.2196/16696
  78. Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: Building an international consensus for the reporting of behavior change interventions. Ann Behav Med 2013; 46(1): 81-95. https://doi.org/10.1007/s12160-013-9486-6
  79. Schoeppe S, Alley S, Van Lippevelde W, Bray NA, Williams SL, Duncan MJ et al. Efficacy of interventions that use apps to improve diet, physical activity and sedentary behaviour: A systematic review. Int J Behav Nutr Phys Act 2016; 13(1): 127. https://doi.org/10.1186/s12966-016-0454-y
  80. de Winter JC, Dodou D, Wieringa PA. Exploratory factor analysis with small sample sizes. Multivariate Behav Res 2009; 44(2): 147-181. https://doi.org/10.1080/00273170902794206