Usability Evaluation Scale for Product of Intelligent Homecare based on Retail Consumer

Abstract

Purpose: The number intelligent homecare products are focused on the development of technology, resulting in a lack of realistic environments or requirements for consumers. The purpose of this paper is to define the consumer and context for intelligent homecare products and to develop a usability evaluation scale. Research design, data and methodology: For this study, first, consumer and contexts related to intelligent homecare products were analyzed through literature review. Second, the primary usability evaluation factors were derived for intelligent homecare products by collecting the factors related to usability evaluation and conducting in-depth interviews with experts. Third, the second usability evaluation factors were derived through survey and statistical analysis based on the derived usability evaluation factors. Results: As a result, users of intelligent homecare products were classified as primary users and secondary consumers and six related contexts. The usability evaluation scale was established with four factors - Functionality, Error, Convenience, and Emotion - and 13 items. Conclusions: This study can be the basis for developing and distributing products that meet the consumer environment and requirements related to intelligent homecare products that will contribute to securing the competitiveness of companies and developing the technology and service value of related industries.

1.Introduction

The smart home is evolving into an intelligent smart home by combining artificial intelligence, machine learning and big data technologies beyond the simple concept of IoT (Ni, 2015). In the United States alone, in 2018, predictions point to 41.2 million smart home devices. Moreover, according to Statista, by 2022 there will be 216.9 million homes worldwide with at least one smart home device (Gomes, Sousa, Pinto, & Vale, 2019). Smart speakers, home electronics, home healthcare products, smart kitchens, ighting control systems, and smart furniture are leading the growth of the smart home market.

The technologies related to the smart home market are developing in the high speed internet, smart phone-based remote controls, home networks, and IoT-based context- aware areas (Lee, 2019). Beyond remotely controlling devices, it is also evolving into an intelligent homecare format where individual devices analyze data such as user location information and lifestyle patterns and recommend the most appropriate service (Andre, Hartson, & Williges, 2003).

In a sociocultural aspect, the low birth rate and increased average life expectancy have led to an aging society and the increased number of single-person households has led to independent living. In addition, due to the increase in the number of dual income couples, there is an increasing demand for home care to help households and reduce household work. Also, there is a demand for intelligent homecare to satisfy the consumer's desire to appease loneliness or to prepare for safety (Yang & Ju, 2012).

Currently, intelligent smart home products are emerging, and the difference of products between domestic and foreign companies is decreasing due to a narrowing of the technology gap and modularization of parts (Fassnacht & Koese, 2006). In addition, there is a lot of space for improvement due to different platforms or security issues due to lack of reflection of realistic environments or consumer requirements as it is focusing on technology in this early stage of intelligent homecare products. Therefore, in line with the quantitative increase in intelligent homecare products, consumer-oriented improvements and designs are necessary for quality growth (Rauschenberger, Cota, & Thomaschewski, 2013).

This paper aims to define 1) the consumer and the context of the consumer in the intelligent homecare industry, and based on this 2) establish a usability evaluation scale for intelligent homecare products. It will be used as a product development guideline of related companies to secure competitiveness and contribute to the consumer-oriented direction of the development of the intelligent homecare industry.

2. Intelligent Homecare Product Consumer

2.1. Intelligent Homecare Product

Smart home, the basis of intelligent homecare, refers to improving the quality of life by providing various information and values to users or residents without the limitation of space and devices through ubiquitous-based integration of information and communication technology into a residential environment (Kwahk & Han, 2002). It is also called Smart Homecare or Connectivity Home (Moreno, Ruiz, Hernadez, & Linden, 2016). It consists of residential space, the physical factors in it, the smart factors that each factor has, and the factors that connect each factor (Jin, Cho, & Ji, 2007). Intelligent homecare refers to a more advanced form of smart home, in which one-to-many or many-to-many connection factors are expanded by maximizing IoT-based connection factors (Mohammadi, Al-Fuqaha, Sorour, & Guizani, 2018).

Intelligent homecare can be divided into electronic products and components, networks, big data and AI, platforms, services, and security according to the industry classification system. In addition, it can be divided into household, health, environment, safety, and leisure according to the value of the service provided. Also, the values that a residential space gives users can be divided into four categories, convenient life, safe life, economic life, and happy life (Martins, Rosa, Queiros, Silva, & Rocha, 2015). These classification points of view are not independent but are closely linked or convergent. Intelligent homecare products, which are an industry field of electronics, have the value of providing various services such as housework, health, and leisure. In addition, they are often linked together on a single platform or work together in combination.

Intelligent homecare products require the integration of intelligent and human factors and understanding of the needs of each user, considering that the needs of the consumer can be reflected depending on the situation (Cho, 2010). The correct solution should be suggested accordingly. Therefore, it includes technology that continuously learns the experiences necessary for consumers to integrate reactions or habits that are now known according to individual capabilities, expectations, and usage situations in the residential environment (Dey, 2001). These intelligent homecare products have the characteristics of Monitoring, Control, Optimization, and Autonomy (Sundaravadive, Kougianos, Mohanty, & Ganapathiraju, 2018).

2.2. Consumer for Intelligent Homecare Product

The connectivity, flexibility, practicality, and necessity to create intelligent homecare are all based on user that is consumer-oriented values (Cho, 2019). Therefore, intelligent homecare should eventually be developed centered on the values desired by the consumer reflecting the user's contests and needs analysis rather than technology (Han, Yun, Kim, & Kwahk, 2000).

Consumers associated with intelligent homecare products can be classified as primary or secondary users. A primary user refers to a person who lives in, contacts, or makes use of intelligent homecare. Basically, it means an individual or a family living in a residential space. Each individual can be linked to intelligent homecare, as can a group of family members. In addition, family members may be connected to each other or their interaction may be made through an intelligent homecare product. Secondary users are users of extended concepts. They are visitors or pets who are partially or temporarily in contact with intelligent homecare other than the primary user.

2.3. Consumer Context for Intelligent Homecare Product

The user's context defined above refers to all of the user's factors when using the product or the platform associated with the product system (Day, 1998). It includes who, when, where, what, how, and why (Brown, Bovey, & Chen, 1997). Table 1 shows the factors of these contexts based on the characteristics of intelligent homecare products. It can be classified into User profile, Physical Environment, Device, Computing Environment, Location, and Time.

Table 1: Context of Intelligent Homecare Product

3. Methodology

3.1. Procedure

In order to develop a usability evaluation scale for intelligent homecare products, the study proceeded in four steps as shown in Figure 1.

Figure 1: Four steps for study

In the first step, usability evaluation factors were collected through literature review. By investigating the usability evaluation status, 70 usability evaluation items were collected, such as general products, web platforms, and digital-based media related to intelligent homecare products. In the second step, duplicate factors were removed from 71 items in step 1 through in-depth interviews of five experts. As a result, 41 items were identified by arranging appropriate factors for intelligent homecare products. In the third step, four factors were derived through survey and statistical analysis on the 41 items. In the fourth step, four factors and 13 items were established based on the statistical analysis results and each factor were defined.

3.2. Primary Factors Extraction

In order to extract the primary usability evaluation factor, the existing usability evaluation factors were first collected for literature review. The factors were collected from ISO (International Organization for Standardization) 9126 software quality evaluation factors, Jakob Nielsen's usability evaluation factors, Peter Morville's Honeycomb evaluation model, the Korea Health Industry Development Institute's aging-friendly product usability evaluation factors, Khan's web-related usability evaluation factors, usability evaluation method of information and communication aids by Son Byung Chang, website evaluation items of Lindgaard, etc.

A total of 70 items were collected and duplicate factors were eliminated through in-depth interviews with five experts. It was reorganized through addition and deletion to suit the usability of intelligent homecare products. In addition, a total of 41 items were extracted by adjusting terms and sentences in accordance with this study scope. The experts for the interview consisted of two researchers from home appliances company, one professor of industrial design, one director of the Emotion Science Center, and one professor of business administration.

3.3. Secondary Factors Extraction

There is a need to identify important factors that users consider important when using intelligent homecare products to develop scales to measure the components of a usability evaluation for intelligent homecare products. To do this, a survey consisting of 41 features from the primary factors extraction was conducted for secondary factors extraction. The survey was conducted on 521 people in their 20s to 60s via online research. The demographic characteristics of the survey participants were 258 males (49.5%) and 263 females (50.5%). The average age was 37.1 years old (sd=10.34), 154 in 20s (29.6%), 175 in 30s (33.6%), 116 in 40s (22.3%), 64 in 50s (12.3%), 12 in 60s (2.3%). The education level showed 92 participants were high school graduates (17.7%), 366 university graduates (70.2%), and 63 graduate school or higher graduates (12.1%). For the use of intelligent homecare products, 263 participants had experience (50.5%) and 258 had no experience (49.5%).

Survey participants responded to the importance of 41 features extracted from primary factors extraction. The 7 points Likert scale was used ranging from 1-totally disagree to 7-totally agree.

In the analysis, first, in order to figure out the responses of the participants to all items and to confirm whether the systematic measurement errors are, we analyzed the mean and standard deviation of each items. Second, we analyzed exploratory factor analysis (EFA) to identify the sub-factors of usability evaluation for intelligent homecare products and tested reliability of items under each factor. Third, we conducted confirmatory factor analysis (CFA) to confirm the construct validity. Lastly, to test nominological validity, we analyzed the correlation between scale, behavioral intention, and innovativeness. SPSS 24.0 was used in analysis.

4. Results

4.1. Means and Standard Deviations of Items

Before conducting factor analysis, means and standard deviations analysis were conducted for examining the variation of responses for the measured items. As shown in Table 2, means of all items ranged from 4.80 to 5.50 and standard deviations ranged from 1.03 to 1.40. It is acceptable as a measurement of usability evaluation for intelligent homecare products in terms of semanticity and discrimination.

Table 2: Means, Standard Deviations of all items​​​​​​​

4.2. Exploratory Factor Analysis

An EFA of 41 items used in this study was conducted to identify factor structure. The principal component analysis was performed with Varimax orthogonal rotation, and factor extraction criteria were set to eigenvalue 1 or higher. As a result, 4 factors were extracted as shown in Table 3. Factor 1 had 12 items and the reliability coefficient was .95. Factor 2 had11 items and reliability coefficient was .95. Factor 3 had 12 items and reliability coefficient was .95. Factor 4 had 6 items and reliability coefficient was .84. By combining the contents of existing research and items, factor 1 was named ‘Functionality’, factor 2 ‘Error’, factor 3 ‘Convenience’ and factor 4 ‘Emotion’. 'Functionality' was defined as the optimized functions of intelligent homecare products to increase the value of users' lives. 'Error' was defined as matters related to errors such as failure or problem of hardware or software of intelligent homecare products. 'Convenience' was defined as convenience and positive characteristics of the environment and condition wherein users use intelligent homecare products. 'Emotion' was defined as a feeling of stimulation or changes to the five senses that appear in intelligent homecare products and are deeply related to subjective satisfaction.

Table 3: The Results of EFA and Reliability Test

Note: KMO=.98, Bartlett’s χ2=18110.70(p<.001)​​​​​​​

4.3. Confirmatory Factor Analysis

CFA was conducted to confirm the factor structure identified by the EFA and to examine the construct validation. Maximum Likelihood Estimation was used for calculation. For CFA, we selected three items each for ‘Functionality’, ‘Error’, and ‘Emotion’, and 4 items for ‘Convenience’ as items representing the meaning of each factor and that do not overlap between items. The reliability coefficients were Functionality .85, Error .89, Convenience .86, and Emotion .72. In all cases, they were above .6 securing internal consistency

The goodness-of-fit of the model for X2 was relatively high as 197.47 (df=57, p<.001) as shown in Table 4, but the overall model fit were statistically satisfied with acceptable level: GFI=.94, AGFI=.91, CFI=.97, NFI=.95, RMR=.05, and RMSEA =.07. In addition, when comparing the fourfactor model with the single-factor model, the four-factor model was more suitable than the single-factor model in all the goodness-of-fit indexes.

Table 4: The Results of CFA​​​​​​​

4.4. Nomological Validity Test

In order to verify the nomological validity of the developed scale, a correlation analysis was conducted between 4 factors, behavioral intention and innovativeness. We measured behavioral intention by adopting the 2-items scale from Escalas and Luce (2003: How likely is it that you would buy this product? How likely are you to use this product?). Cronbach’s α was .91. To measure individual’s innovativeness, we used following 7 items (Parasuraman, 2000): 1. Other people come to you for advice on new technologies. 2. It seems your friends are learning more about the newest technologies than you are(r). 3. In general, you are among the first in your circle of friends to acquire new technology when it appears. 4. You can usually figure out new high-tech products and services without help from others. 5. You keep up with the latest technological developments in your areas of interest. 6. You enjoy the challenge of figuring out high-tech gadgets. 7. You find you have fewer problems than other people in making technology work for you. Cronbach’s α was .86.

As a result of the analysis, as shown in Table 5, the relationship between behavioral intention, innovativeness and four factors was found to be positive(+). In particular, the greater the importance of functionality, convenience, emotion, and error considered, the higher the behavioral intention of the intelligent homecare product. The higher the individual's innovativeness, the greater the importance placed on the functionality, convenience, emotion, and error.

Table 5: The Results of Correlation Analysis

** p<.01​​​​​​​

These results indicate that the scale developed in this study proves the nomological validity

5. Discussion and Conclusion

This paper defined the user, the relevant consumer, and context for the development of a usability evaluation scale for intelligent homecare products and established four usability evaluation factors and 13 items.

First, intelligent homecare products have the characteristics of monitoring, control, optimization, and autonomy by maximizing IoT-based connection factors, and aim to provide services to increase the value of users' lives. Users could be classified as primary users, individuals, or families living in residential spaces and secondary users, which is an extended concept. Six relevant contexts were defined, User Profile, Physical Environment, Device, Computing Environment, Location, and Time. This definition enabled a systematic analysis and understanding of characteristics for targeting new development areas of intelligent homecare products.

Second, four elements-functionality, error, convenience, and emotion-and 13 items were developed as usability evaluation factors for intelligent homecare products. Functionality refers to the various functions of intelligent homecare products that are optimized to increase the value of users' lives and consists of three items. Error is related to malfunctions such as failure or problem with the hardware or software of an intelligent homecare product and defined as three items. Convenience is the comfortable and positive characteristics for the situation, environment, and conditions where users to use intelligent homecare products and consisted of four items. Emotion is a sensation or change in the five senses from intelligent homecare products. It is defined as three items that are closely related to subjective satisfaction.

Third, the usability evaluation scale established in this study can be used to suggest the correct direction and guidelines for intelligent homecare products. As the development and sales of intelligent homecare products are in the early stages, there is a lack of guidelines, standardization, and user experience analysis. The usability evaluation scale developed in this study presents the evaluation guidelines for products and will contribute to the development of intelligent homecare products and technology application. It can be used to secure the competitiveness of related companies as a foundation for systematically evaluating and analyzing consumer environment and needs. In the long term, it can be used to develop the value of intelligent homecare product service.

Since the subject of this study was intelligent homecare products, which are still in the early stages in the market, it was difficult to collect experienced samples because of the low penetration rate. The limitation of the study was that it included both experienced and inexperienced subjects. In addition, since repeated surveys in scale development can reaffirm the reliability of the scale, it is necessary to supplement these limitations and continue research to reconfirm the reliability of the scale through revalidation in the future. Lastly, it is necessary to test prediction validity of the scales presented in this study with actual user data such as usage frequency of intelligent homecare products in future research.