Design and Implementation of a Smart Glass Application for XR Assisted Training of Core Nursing Skills

Abstract

Extended reality-assisted training offers repeatable learning opportunities at a low cost. This paper proposes a smart glass application for training core nursing skills to nursing students, who often need to memorize and practice training sequences. The proposed smart glass application interactively presents a series of instructions to help students remember and perform two core nursing skills in the correct order. We conducted a usability test on 30 undergraduate nursing students in their third year using the smart glass application. Our initial findings show that many students have positively evaluated the possibility of using smart glasses for training, but have also encountered several challenges with the smart glass application's user interface, which takes time to adapt.

Ⅰ. INTRODUCTION

Educational platforms using extended reality (XR), including virtual reality and augmented reality, are attracting attention. Recently, as COVID-19 necessitates non-face-to-face classes, many university students attend classes using virtual platforms such as Zoom, Skype, and Google Meet. As an alternative to non-contact education, there is a growing demand for non-face-to-face education using virtual reality and augmented reality technologies. However, the current non-face-to-face technology has limitations regarding conducting experiments, training, and practice. Practical training is a necessary process to receive safety training prior to fieldwork.

Many researchers have studied education platforms with XR, including virtual, mixed, and augmented reality. The reasons for adopting XR technology for learning include student safety, repeatable learning, reduced educational costs, and overcoming spatio-temporal limitations. Various types of user interfaces such as EEG devices are implemented for brain-computer interaction [1]. XR-assisted training is used in diverse domains, including military training, medical training, government officials, school education, and manufacturing.

XR technology could benefit nursing education, assisting nursing students in training with real-time supplementary information. The requirements of nursing education differ from those of manufacturing assembly. Nursing students conduct many complicated and intermingled tasks, such as patient interaction and equipment assignment; conforming to the order of procedure is crucial to these tasks. For example, there is a standardized process for blood transfusions that causes severe harm to the patient if it is incorrect or modified. Therefore, it is necessary to develop a suitable platform to enable the performing of sequential and complex tasks.

This paper proposes a smart glass application for core nursing skills to train nursing students. We have designed and developed a core nursing skills application for smart glasses in accordance with the requirements of nursing academics. Graphical instructions for different skills are listed and displayed on the screens of smart glasses. A student wearing smart glasses can choose a skill, practice it, and simultaneously memorize the sequence. Using the developed prototype, we conducted a usability test for undergraduate nursing students at our university.

Ⅱ. BACKGROUND KNOWLEDGE

2.1. Smart Glasses

Smart glasses are head-worn mobile computing devices, which contain multiple sensors, processing capabilities, and optical head-mounted displays [2]. Google Glass first appeared in 2014, opening the era of smart glass [3]. Since the launch of Google Glass, various types of smart glass have been released: Epson’s Moverio BT-300, Vuzix Blade [4], and Google Glass Enterprise Edition. These smart glasses are lighter than virtual reality HMDs. Besides, these have a monocular display for displaying information and provide touch or voice input functions.

Since smart glasses have not yet been popularized as they are in their infancy, many people are not familiar with smart glasses' operation. Most smart glasses provide input methods such as touching the eyeglasses, gestures using head movement or various input devices, and natural voice recognition. In addition, since smart glass input device standardization has not yet been achieved, the same gestures can be interpreted differently depending on the device.

2.2. Training and Practice with Smart Glass

According to recent research, smart glasses are expected to be used in medical fields, manufacturing, and education. A remote collaboration system using smart glass has been developed in the manufacturing industry, and a system that solves problems in the field through collaboration with experts from a distance has been developed [5].

Grunerbl et al. proposed an initial experimental design for nurse emergency training to improve and understand the resuscitation skills of student nurses undergoing medium-fidelity simulation emergency scenarios [6]. In this experiment, nursing students wear Google Glass while attending the SimMan-patient, a computerized mannequin [7]. Their experiments have shown the viability of using smart glass together with on-body sensors in emergency training scenarios.

Nurse watch [8] is a smartwatch system developed in collaboration with a team of nurses working in a Swedish ICU. This system provides real-time monitoring of vital signs, threshold alarms, and to-do reminders. In addition, the Kanban board, visualized on a multitouch screen, provides an overview of completed and upcoming tasks. The system was perceived by nurses to be useful in an ICU environment.

2.3. Core Nursing Skills in Korea

In view of the growing need for better practical skills among registered nurses (RNs), the Korean Accreditation Board of Nursing Education (KABONE) has proposed 20 core nursing skills that all nursing students must master [9]. Each skill consists of 2–30 sequential tasks. For example, blood transfusion has about 23 subsequent tasks, including checking patients and informed consent, washing hands, preparing tools and vital sign monitoring. The critical point for learning core nursing skills is that students must accurately remember and execute the sequence. The core nursing skill course is important because the effects of medical carelessness may lead to adverse consequences such as infections, which can be life-threatening for a patient.

Ⅲ. DESIGN APPLICATION FOR SMART GLASSES

The goal of the study is to enable nursing students to self-practice core nursing skills. In general, students write sequential instructions on paper to memorize and practice them. It is not easy to focus on the practice because the students’ hands and gaze are preoccupied with the paper. It is necessary to make students’ hands-free, engage them in practice to help them remember the correct order of practice and provide intuitive graphical guidance to enable accurate performance. Since text-based information is dependent on each student’s interpretation ability, image-based instructions are better alternatives to help grasp and understand the educational content in real-time.

We redefine several design factors of smart glasses using smart glass features discussed earlier, examples of education using smart glass, and core nursing skills as background knowledge. There are 18 design factors of smart glasses proposed by N. M. Kumar [10]. To be compatible, the smart glasses should be linked to the hospital network and synchronized with several medical devices. Furthermore, it is essential to incorporate data security that shows patient information only to nurses or doctors who only need it. It is crucial that these smart glasses be immune to medicines used in hospitals, be waterproof, and not be contaminated with dust or bacteria. This condition requires a hands-free function so that the glasses are not touched. Voice control is a quick and convenient way to operate without using touch.

Therefore, we chose a smart glass application for training core nursing skills through hands-free and graphical guidance. When a student wears a smart glass, it shows a sequence of skills selected as core nursing skills. One skill is composed of 20 to 30 image sequences. The images show the tasks in detail. When the student wants to see the next task, he/she gives an input command to show the next image. However, we could not choose a voice command interface because of the awake call ‘hello Vuzix’ every time before giving a voice command.

Ⅳ. IMPLEMENTATION

We developed an image guidance program for two core nursing skills, including blood transfusion and intradermal injection. The numerous steps of these skills were split into several graphical images to be displayed on a smart glass. In other words, when students wear smart glasses and select the skills they want to practice among the core nursing skills, graphical images appear sequentially and can be followed by memorizing the sequence. Graphical guidance is composed of a sequence of images that show tasks in nursing skills. The images were selected by a professor working in the nursing department of our university. Fig.1 shows the slide number, images, and text to be used sequentially by nursing students.

Fig. 1. Example of graphical guidance.

The user interface of the Vuzix Blade allows UI elements to be navigated with simple left/right/up/down navigation. The menu is expressed in a square shape at the bottom of the screen, as shown in Fig.2. Students can select a submenu by swiping the touchpad. When a student chooses a sub-menu, the image corresponding to that is displayed. The student can swipe the Vuzix touchpad to see the next slide. When they want to move to the upper menu, they tab the touchpad using two fingers.

Fig. 2. User interface. (a) main menu of our application. (b) one of the images.

Ⅴ. EXPERIMENTAL RESULTS

Thirty third-year undergraduate students participated in our study. We requested them to use smart glasses for the practicing of blood transfusion and intradermal injection among core nursing skills. Two hours were allocated to two students for simultaneous practice. In this study, 30 students participated in two days.

Five and two items were used for the quantitative and qualitative usability test, respectively, as shown in Table 1 and Fig. 3. As shown in Table 1, most of the answers are positive, and the mean is approximately 8. The reason the score of item three is lower than others is that the smart glass resolution was low (480 x 853), and students with poor eyesight had trouble reading the text. When students see images on the screen, it is relatively easy to recognize at a glance. Still, in the text that appears with the image, it is difficult to recognize because it is relatively small and overlaps with the image.

Table 1. Result of the survey

Fig. 3. Two Questionnaires. a) Q1 is “What is your overall feeling for our application ?”, b) Q2 is “Have you encountered any difficulties during the experiment?”

Thirteen students expressed their experiences (Q1) as “wonderful,” while eleven students mentioned it as “convenient” (Fig. 2(a)). They think the proposed system helps memorize the task sequence. However, some students had difficulty wearing the device, and some students focused on familiarizing themselves with the new device rather than developing skills. Some students had bad eyes, so they couldn't see while wearing smart glasses. Regarding inconveniences (Q2), nine students pointed out the difficulty when operating the unfamiliar touchpad of the Vuzix blade (Fig. 2(b)). Because they never used the glass before, so some students don’t know where to touch it. There were also reports and complaints regarding blurry text, eye fatigue, and wearability.

Ⅵ. DISCUSSIONS

Through this experiment, we created an improvement list of the smart glass UI. First, there is a need to simplify user interaction using a touchpad. Since students were not familiar with smart glasses, there was confusion while performing even simple operations. Considering the complexity of nursing skills, users could easily be annoyed by these newly adopted devices. Minimizing user efforts for the learning of smart glasses would result in promising outcomes.

Second, the image that introduces the task must deliver messages clearly so that it can be intuitively understood. In addition, it should be possible to quickly and intuitively understand by minimizing the length of the text. Finally, it is necessary to select a color that does not cause eye strain. The information would better be conveyed via smart glass with user-friendly design.

The limitation of the proposed system is the user input of the smart glass. A student should use the touchpad on the smart glass to see the next slide. While they are doing one of a task in core nurse skill, they have to use both hands to manipulate their tools. That is cumbersome. We could find a context-aware approach to solve this problem.

Ⅶ. CONCLUSION

This study demonstrated a high potential for the integration of XR technology in skill training for nursing professionals. It was challenging to facilitate all students to adapt to the XR learning environment owing to unfamiliarity with the smart glass device. Other challenges, such as low display resolution, narrow field of view, and limited visibility of the text, could impede the active use of smart glasses while performing tasks in practice. In future work, we plan to develop proper smart glasses for nursing students, conduct usability tests, and prove our application’s effectiveness in educating nursing students.