Particle Swarm Optimization Using Adaptive Boundary Correction for Human Activity Recognition

Abstract

As a kind of personal lifelog data, activity data have been considered as one of the most compelling information to understand the user's habits and to calibrate diagnoses. In this paper, we proposed a robust algorithm to sampling rates for human activity recognition, which identifies a user's activity using accelerations from a triaxial accelerometer in a smartphone. Although a high sampling rate is required for high accuracy, it is not desirable for actual smartphone usage, battery consumption, or storage occupancy. Activity recognitions with well-known algorithms, including MLP, C4.5, or SVM, suffer from a loss of accuracy when a sampling rate of accelerometers decreases. Thus, we start from particle swarm optimization (PSO), which has relatively better tolerance to declines in sampling rates, and we propose PSO with an adaptive boundary correction (ABC) approach. PSO with ABC is tolerant of various sampling rate in that it identifies all data by adjusting the classification boundaries of each activity. The experimental results show that PSO with ABC has better tolerance to changes of sampling rates of an accelerometer than PSO without ABC and other methods. In particular, PSO with ABC is 6%, 25%, and 35% better than PSO without ABC for sitting, standing, and walking, respectively, at a sampling period of 32 seconds. PSO with ABC is the only algorithm that guarantees at least 80% accuracy for every activity at a sampling period of smaller than or equal to 8 seconds.

1. Introduction

Personal life information is currently being treated as an important resource for more accurate diagnoses. Practitioners need routine data, as well as vital data, to see patient’s life patterns for precise, predictive, and personalized treatments. For this reason, there have been many studies on using routine data to help practitioners. Ito et al. [1] tried to overcome the limitation of medical information by gathering voluminuous and varied lifelog data for medical purposes. Heo et al. [2] developed an Android-based smartphone application to encourage breast self-examination in daily life to find breast cancer at an early stage. Kwon et al. [3] built a lifelog agent based on Health Avatar Platform [4] to summarize human activity patterns using smartphone sensors in order to incorporate lifelog data into a medical record.

Fig. 1.System dialog for medical diagnosis and treatment with lifelog in NTT [1]

A lifelog is all information that can be collected from a person, such as sleep information, exercise information, and eating habits. A lifelog can be collected from a variety of devices such as smartphones or wearable sensors, and exploited for various purposes, including human activity recognition. For instance, a digital health screening form [5-6] requires lifestyle in-formation such as activity, sleeping, exercising, and eating habits, as well as entries in a pa-per-based health screening form. If questions such as “how long do you walk per day,” “how long do you sit at a desk each day,” or “how often do you exercise” can be filled in with ac-tivities automatically derived from the lifelog data, it is then possible to obtain the objective lifestyle information. Hence, practitioners can easily achieve a more accurate diagnosis using a digital health screening form.

In particular, activity data are one of the most fascinating information retrieved from the lifelogs. Activity is actually something that is done at some time (e.g. sitting, walking). Ac-tivity data in daily routines have been used to understand the user’s habits or lifestyle, and have been used in some applications. For instance, Zwartjes et al. [7] developed an ambulatory monitoring system that analyzes current activity of the patient to monitor motor symptoms in Parkinson’s disease. Another example is using activity data collected from smartphone sen-sors to measure the amount of calories burned [3].

In this paper, we choose lifelog data as the sensor values of a triaxial accelerometer in a smartphone, and propose a robust method to sampling rates that recognizes the user’s activi-ties. Since lifelog data are collected over a long period of time, it is necessary to pay attention to actual smartphone usage, battery consumption, or storage use of a smartphone, which highly depends on a sampling rate of the accelerometer, because a high sampling rate leads to high accuracy in activity recognition. Activity recognition methods with well-known classi-fication algorithms, including MLP [8], C4.5 [9], or SVM [10], suffer from a loss of accuracy when the sampling rate decreases. Thus it is not suitable to implement activity recognition by direct adaptions of typical classification algorithms.

We therefore introduce particle swarm optimization (PSO) [11-12] using adaptive boundary correction (ABC). PSO is a stochastic method that tries to repeatedly enhance a particle (candidate solution) of a given problem. One of the greatest advantages is that in PSO, each particle’s movement is affected by the best known position of the entire group as well as by the best known position of the particle, which makes the particle move to the more optimal posi-tion. We start from PSO for classification, described in [13], that has better tolerance to de-creases in sampling rates. Using PSO for classification, however, may fail to classify some input data for a low sampling rate. The ABC method helps recognize the unclassified points by adjusting the classification boundary based on the shape of the input data.

The rest of this paper is organized as follows. In section 2, the related work on human ac-tivity recognition and particle swarm optimization is summarized. Section 3 describes the ABC technique for activity classification. Section 4 provides an analysis of accelerations to see the properties of each activity. The experimental results are then shown in section 5, and we offer some concluding remarks in section 6.

 

2. Related Work

2.1 Human activity recognition

Human activity recognition detects the person’s activities with an analysis of the acceleration data. It has been studied in computer vision, wearable computing, data mining, and their applications. Although there are some studies [14-16] to detect micro activities with vision systems, we concentrated on accelerometers that are small, light-weight, and embedded in a large number of portable devices. Since accelerometers measure the change in a person’s body, they play a significant role in activity recognition. To determine the human activities properly, at least one accelerometer must be attached to the body so that acceleration data of the person are recorded during certain activities. The acceleration data are converted into statistic data after feature extraction, and are then applied to algorithms for activity recognition.

Some applications apply human activity recognition for several purposes. Jafari et al. [17] used accelerometers to perceive the urgent states of elderly users. These allow others to be alerted when such users face a potentially dangerous situation. The authors in [18-20] ex-ploited activity recognition for activity monitoring, and transform exercise information into user-friendly information after classification. Kwon et al. [3] incorporated the activity recog-nition with healthcare service platform to provide an application to show the user’s activity patterns in a graph.

Several studies have introduced algorithms and models for activity recognition. Stikic et al. [21] used RFIDs attached to objects used at home, along with accelerometers, to record what the person is doing and what object is used each time. Some studies [22-23] have developed algorithms for human activity recognition, each of which is concentrated on a distinct col-lecting method or different target activities. Uddin et al. [24] used body joint-angle features and a hidden Markov model for activity recognition with a single camera.

2.2 Particle swarm optimization

Particle swarm optimization (PSO) is one of the best algorithms among swarm-based approaches that are reasonable ways to solve a number of difficult problems in nature. PSO, inspired by flocks of birds and schools of fishes, is a stochastic method that iteratively improves a candidate solution of a given problem to find the optimal solution [11-12]. In PSO, every particle shares its social experience derived from traveling. Depending on the collective experience, the entire group moves to the optimal position.

In PSO, each particle computes its next velocity using the states of the other particles, and then computes its next position. After the particle changes its position, it shares its position with the others, helping them determine their next positions. The process is repeated until an optimal position is found.

The velocity and next position of a particle can be computed through equations (1) and (2), where vt and xt indicate the velocity and position of the particle at time t, respectively. In this paper, constants w,c1, and c2 are chosen to be 1, 2, and 2, respectively. In addition, rand() generates a real number within [0, 2], pBest is the best-known position of a particle, and gBest is the best-known position of the entire group. PSO moves the particles by iterating equations (1) and (2) to find the optimal position of the group. PSO is finished when the termination criterion is met, for instance, PSO is repeated for a given number of times, or until the next velocity is zero.

 

3. Adaptive Boundary Correction

3.1 Data classification using a boundary

In classification, the input data can be assigned to a class using a set of representative positions of each class, which can be determined by PSO. To perform classification quickly and accu-rately, we first create the classification boundaries of each class, as described in [13]. For each feature, the boundary of a class is computed using the equations (3) and (4).

where xi is the i-th feature value of a representative position, rand() generates any number within [0,1], Max i is the maximum value among the i-th feature values, and Mini is the minimum value among the i-th feature values. After creating the boundary, one can determine whether a given point is a member of the class using the boundary, as shown in Fig. 2.

Fig. 2.Flowchart of particle swarm optimization

Fig. 3 describes a pseudo code of data classification using a boundary. For each feature value of a given point, if it is not included in the boundary, we then neglect the point because it is not a member of the class. If all of the feature values are included in the boundaries, we can conclude that the point is a member of the class.

Fig. 3.The pseudo code of data classification using a boundary

3.2 Challenges of classification using a boundary

Classification using a boundary has the advantage of being executed quickly and accurately. However, the method has some drawbacks. Since the feature values are likely to vary when a sampling rate becomes lower, the test points may not be fit in the boundary of the class generated by the training data, even though they are actually members of the class. For instance, consider that a class is distributed within [1,10] and its representative point is 5. From equations (3) and (4), the boundary of the class can be computed as LowerBound = -4 and UpperBound = 14 (if rand() = 1). If the test data are positioned within [-5,15], a point out of [1,10] should be determined as not being a member of the class. Since the point is the actual member, this situation causes a classification error. Another example is shown in Fig. 4. Although point F should be a member of class B, the algorithm does not assign F to class B because it is located outside the boundary of class B.

Fig. 4.Example of a point outside the boundary of a class

Points outside the boundary are sometimes found during activity classification. In general, the training data are collected intentionally under noiseless conditions. The test data, however, are recorded under diverse and noisy conditions. In particular, the sampling rate of the ac-celerometer plays a significant role in activity recognition [25]. Since these points are not able to be classified, they cause a decrease in the classification accuracy. A method to cope with such points to reduce the classification errors should be derived.

3.3 Adaptive boundary correction (ABC) approach

A naïve approach to solve the out-of-boundary problem is used to expand the boundary of each class. As shown in Fig. 5, however, the extension of the boundaries may cause an intersection of the regions of two classes. The intersection may cause another problem, i.e., which cluster the point in the overwrapped region is assigned to.

Fig. 5.Problem of a naive approach

We propose an Adaptive Boundary Correction (ABC) approach to solve the out-of-boundary problem without overwrapped regions. The ABC approach copes with sin-gular points outside the boundaries of every class by applying another method to increase the classification accuracy. In this paper, we used the distance from each representative point to the singular point. Since the representative points are known after the PSO phase, we can compute the distance from each representative point, and can find which representative point is the closest one. The singular point can then be determined as a member of the corresponding class that has the closest representative point. Fig. 6 shows the flow of the ABC approach.

Fig. 6.Flowchart of the ABC approach

For instance, assume that there is a point F that is assigned to no group, as F is not within the boundary of any class, as shown in Fig. 7. According to the ABC approach, to assign F to one of the three classes, we first need to compute the distances as follows:

where i is the number of features, F=(f1,f2,⋯,fi), and the representative points of each class are given as A=(a1,a2,⋯,ai), B=(b1,b2,⋯,bi), and C=(c1,c2,⋯,ci). Compar-ing these distances, we can find the closest class, B. Hence, we determine that F is a member of the class B.

Fig. 7.Example of ABC approach

 

4. Analysis of Activity Data

4.1 Data collection

To evaluate the PSO using ABC for activity recognition, we collect the activity data in real life. Since a smartphone has a built-in triaxial accelerometer, this experiment exploits a smartphone by implementing an application that handles the sensor. A smartphone was kept in a pants pocket to collect triaxial acceleration during a series of activities. The sensor data were rec-orded in a database and used to verify the effectiveness of the proposed algorithm.

Fig. 8 shows the position of the smartphone and axes of the accelerometer relative to the user. The x axis implies the horizontal movement of the user, the y axis implies the vertical movement, and the z axis implies the movement of the leg. The unit of acceleration is m/s2.

Fig. 8.Direction of the accelerometer

4.2 Properties of the activities

Before analyzing the sensor data of a user, we need to determine activities to be recognized. In general, the basic human activities during a daily routine are walking, sitting, and standing. To describe the formal description of each activity, we referred to the definitions in [26]. The formal definition of the three activities are presented as follows:

These activities are mutually exclusive, and are considered to be the most basic activities in many studies. To identify these activities, it is necessary to know the properties of each activity that are exploited during the activity recognition processes. In this subsection, we analyze the real sensor data and compute the statistics of the activities.

4.2.1 Sitting

Fig. 9 shows a plot of the acceleration signal for sitting. The signal demonstrates that the x, y, and z values are almost constant, and the z values have larger accelerations than any other values. The statistics of each axis are as follows:

Fig. 9.Acceleration signal for sitting

4.2.2 Standing

Fig. 10 shows an acceleration signal for standing. Similar to sitting, the x, y, and z values are almost constant. Since the axes of the accelerometer are changed, the y values, instead of the z values, have larger accelerations than any other values. The statistics of each axis are as follows:

Fig. 10.Acceleration signal for standing

4.2.3 Walking

Fig. 11 shows a graph of the acceleration signal for walking. While the values of sitting and standing are very stable, the values of walking are relatively varied. The y values have larger accelerations than any other values. The statistics of each axis are as follows:

Fig. 11.Acceleration signal for walking

4.3 Feature extraction

According to the properties of each activity and previous work [27-28], we chose the features to be extracted. For a window of accelerations w=((x1,y1,z1),⋯,(xn,yn,zn)) where xi, yi, and zi are the i-th acceleration of x, y, and z directions, the average and standard deviation of accelerations are used. Three average (μx,μy,μz) and three standard deviations (σx,σy,σz) are computed as follows

In addition, we used three features that consisted of the energy of a signal that the Discrete Fourier Transform generates from the original signal. If Xj is the Discrete Fourier Transform of xi, the energy of each direction is computed as follows:

Thus, nine features, consisting of the average, standard deviation, and energy of three di-rections, are used for PSO using ABC. Table 1 shows an example of the feature vectors for each activity.

Table 1.Example of feature extraction

 

5. Experiments

In this section, we evaluate the activity recognition algorithm using the ABC technique, and compare our approach to PSO without ABC, as well as other algorithms such as MLP [8], C4.5 [9], and SVM [10]. To alleviate the computational complexity of PSO, we adopted intelligent dynamic swarm [29] that was shown to be 30% faster than traditional PSO. For training dataset, we intentionally collected the accelerations from a triaxial accelerometer, which is embedded in the smartphone during three activities, each of which was performed for five minutes, at a sampling rate of 16 Hz. For test dataset, we collected accelerations in real life for two hours, with sampling periods (=(sampling rate)-1) of 2-4, 2-2, 1, 4, 8, 16, and 32 seconds. To compute the accuracy, the intervals of accelerations were manually labeled as one of the three activities. We used an Android-powered smartphone with an application implemented to collect accelerations, as shown in Fig. 12.

Fig. 12.Application for data collection

5.1 Evaluation for sitting

Fig. 13 shows the experiment results for sitting. Since the acceleration signal is very stable for sitting, the experiment shows that all methods recognize sitting with high accuracy. When the sampling period is shorter than or equal to 4 seconds, the accuracy of each algorithm is larger than 98%. If a sampling period increases, then the accuracies drop significantly. For instance, PSO without ABC, MLP, C4.5, and SVM show 97.7%, 94.8%, 94.0%, and 96.8% accuracies at the sampling period of 16 seconds, and 92.6%, 89.4%, 87.6%, and 92.9% accuracies at 32 seconds. PSO with ABC, however, shows the best performance; its accuracy is 98.0% even at a sampling period of 32 seconds.

Fig. 13.Experiment results for sitting

5.2 Evaluation for standing

Fig. 14 shows the experiment results for standing. Since the number of accelerations for standing is much smaller than that for sitting, the results looks unstable. Still it verifies that PSO with ABC is better than the other methods. At the sampling period of 4 seconds, most of algorithms are comparable in accuracy. When the sampling period increases, the accuracy of PSO with ABC slightly increases at a sampling period of 8 seconds while the accuracy of PSO decreases. Especially, PSO with ABC and PSO without ABC show 80.9% and 79.2% accuracy at the sampling period of 4 seconds, respectively. When the sampling period is 8 seconds, PSO with ABC shows a robust accuracy of 83.6% while PSO without ABC shows a declined accuracy of 74.1%. This is because ABC catches unclassified data that are generated due to lower sampling rates. Therefore, this experiment shows the effectiveness of the ABC approach.

Fig. 14.Experiment results for standing

5.3 Evaluation for walking

Fig. 15 shows the experiment results for walking. The walking results are relatively unstable because walking is a dynamic activity. Nonetheless, the accuracies of all the algorithms are larger than 85% when a sampling period is smaller than or equal to one second. If a sampling period increases, then the accuracies drop significantly. For instance, PSO without ABC, MLP, C4.5, and SVM show 89.2%, 98%, 97.2%, and 96.4% accuracies, respectively, when the sampling period is one second. The accuracies, however, become 69.8%, 90.9%, 87.9%, and 72.7% at the sampling period of 8 seconds, and become 50.0%, 66.7%, 77.8%, and 44.4% at 32 seconds. The accuracy of PSO with ABC, however, is less reduced than that of the others. The accuracy is 89.2% at one second, 86.5% at 8 seconds, and 85.0% at 32 seconds. Especially PSO with ABC outperforms the other algorithms when a sampling period is 32 seconds; its accuracy is at least 7% better than the others’.

Fig. 15.Experiment results for walking

5.4 Discussion

The experimental results show that the accuracy of the PSO with the ABC approach is better than that of PSO without ABC and the others when a sampling period is relatively large. The ABC approach improves the overall performance by dealing with the outliers that are not in the boundary of the class. For instance, PSO with ABC shows 6%, 25%, and 35% better results than PSO without ABC for sitting, standing, and walking, respectively, with a sampling period of 32 seconds. Especially PSO with ABC is the only algorithm that guarantees at least 80% accuracy for every activity at a sampling period of smaller than or equal to 8 seconds.

Table 2 shows the average activity recognition results of all sampling periods for each al-gorithm with respect to each activity. Since sitting data are relatively simple, every algorithm accurately identify “sitting” for every sample period. For standing, PSO with ABC outper-forms the others because PSO with ABC recognizes “standing” consistently even when a sampling period is large. For walking, PSO with ABC is comparable to MLP and C4.5 and outperforms PSO without ABC and SVM.

Table 2.Overall average activity recognition results for each algorithm

Table 3 shows the average accuracy differences between the successive sampling periods for each algorithm. The values indicate the tolerance to changes in the sampling periods without a steep decrease in the activity recognition accuracy. The results show that PSO with ABC is the most tolerant algorithm that can recognize activities without losing too much accuracy.

Table 3.Average accuracy differences between the successive periods for each algorithm

 

6. Conclusions

This paper proposed PSO with ABC to solve the difficulty of human activity recognition due to the fact that the accuracies of activity recognition algorithms, such as MLP, C4.5, and SVM, become lower as a sampling rate decreases. Although a high sampling rate is neccesary for high accuracy, it is not desirable for actual smartphone usage, battery consumption, or storage occupancy. To maintain recognition accuracies, we started from PSO, which is a relatively consistent algorithm in sampling rates, and we proposed PSO with an ABC approach. ABC is tolerant of small sampling rates in that it identifies all data by adjusting the classification boundary of each activity.We evaluated PSO with ABC and other algorithms with lifelog data from a triaxial accelerometer in a smartphone, with sampling periods of 2-4, 2-2, 1, 4, 8, 16, and 32 seconds. The experimental results show that PSO with ABC has better tolerance to declines of sampling rates than PSO without ABC and other classification methods. In particular, PSO with ABC is 6%, 25%, and 35% better than PSO without ABC for sitting, standing, and walking, respectively, with a sampling period of 32 seconds. PSO with ABC is the only algorithm that guarantees at least 80% accuracy for every activity at a sampling period of smaller than or equal to 8 seconds.

As future work, we are planning to examine our method to detect more activities, such as walking up and down, or running, as well as sitting, standing, and walking, while preserving a high precision. We are also considering a way of coping with exceptional cases, for instance, calling or charging, or keeping the device in places other than a pocket, while recording the lifelog data.