Activity and Safety Recognition using Smart Work Shoes for Construction Worksite

Abstract

Workers at construction sites are easily exposed to many dangers and accidents involving falls, tripping, and missteps on stairs. However, researches on construction site monitoring system to prevent work-related injuries are still insufficient. The purpose of this study was to develop a wearable textile pressure insole sensor and examine its effectiveness in managing the real-time safety of construction workers. The sensor was designed based on the principles of parallel capacitance measurement using conductive textile and the monitoring system was developed by C# language. Three separate experiments were carried out for performance evaluation of the proposed sensor: (1) varying the distance between two capacitance plates to examine changes in capacitance charges, (2) repeatedly applying 1 N of pressure for 5,000 times to evaluate consistency, and (3) gradually increasing force by 1 N (from 1 N to 46 N) to test the linearity of the sensor value. Five subjects participated in our pilot test, which examined whether ascending and descending the stairs can be distinguished by our sensor and by weka assessment tool using k-NN algorithm. The 10-fold cross-validation method was used for analysis and the results of accuracy in identifying stair ascending and descending were 87.2% and 90.9%, respectively. By applying our sensor, the type of activity, weight-shifting patterns for balance control, and plantar pressure distribution for postural changes of the construction workers can be detected. The results of this study can be the basis for future sensor-based monitoring device development studies and fall prediction researches for construction workers.

1. Introduction

Construction industry is highly dependent on labor force and construction worker’s safety is directly related to productivity. However, the workers at construction sites are at high risk of getting injured or dying from external causes. Workers at construction sites are frequently exposed to many types of dangers involving falls, tripping, and missteps on the stairs. A study by Kenneth et al. reported that 309,060 cases of injuries were from falls, and that number is higher than the injuries from contact with objects and equipment [1]. According to the Bureau of Labor Statistics (BSL) Survey of Occupational Injuries and Illnesses (SOII) in 2015, falls accounted for 26.8% of all injuries. Courtney et al. documented work-related disability and death due to falls at construction sites and reported that the common cause was the stairs at the work sites that often lack firm surface and railings [2]. Danger detection and warning system for construction work site is crucially needed to reduce and prevent such tragedies.

Industrial safety includes securing the safety of workplace since construction materials, machinery, and devices can cause accidents [3]. Although the importance of managing the safety of construction workers is raised every year, there still are very few studies on this matter. Falling accidents have long been recognized as an important occupational safety issue, because they not only directly affect work speed and performance but also are the main causes of death or injury at the construction sites [4-6]. Stairway falls in general are one of the global health concerns and are often associated with extensive damages and injuries with poor prognosis [7,8]. Previous studies reported that falls from the stairs lead to accidental deaths in older adults and to the coincident activity for 7-36% of falls [9,10]. Most of the fall-related accidents were reported to be in high teens or low twenties [11-17] and appeared particularly evident for the middle-aged adults [15-17]. Since the age range of construction workers is wide, personal protective equipment (PPE) practical for people of all ages is needed.

Traditional types of PPE include safety helmet, worker, gloves, work pants, shirts, and glasses. Among these, wearing safety helmet and safety shoes is essential for workers’ safety. With emerging technology, several studies developed smart safety helmets using accelerometer sensor for fatigue detection and safety shoes with pressure sensors [18,19]. A study by Yang et al. [20] assessed gait in a laboratory to identify fall harzards of construction envrionments using accelerometer sensor and motion capture system.

Other studies tested the effect of safety training and education for construction workers in recognizing and identifying risks [21,22]. However, risk detection relied on the judgment of a subjective and error prone person due to his or her knowledge and experience of various levels of risk conditions. Construction sites are dynamic, adding to the complexity of detecting hazards, especially in low light or noisy conditions. Considering this, additional approaches are needed to address the current limitations to improve risk identification at construction sites. Recent attention has been paid to the researches on the safety monitoring of the operator using wearable pressure sensor, acceleration sensor, and gyroscope.

Some authors focused on detecting a fall risk before it occurs in humans and robots [23-25]. However, these are highly focused in threshold based algorithms, and therefore very subject dependent. To monitor movement execution in real time and to prevent falls during daily life activities, a study by Ribeiro et al. (2019) developed an offline IMU-based classifier capable of distinguishing normal gait from fall and pre-fall situations [26].

Along with various high-functioning and user-friendly sensors, a systematic management system is now needed to monitor and warn the construction workers of the dangers they face at the worksites. Although needed, setting up real-time monitoring systems at construction sites with commercial sensors, motion detecting cameras, wired computers, and general alarming programs would be quite costly and inefficient for the workers. A cost-effective, nonrestrictive sensors and real-time warning system are essential to reduce accidents and life-threatening injuries that occur at construction work sites. Thus, this study developed inexpensive but high-functioning wireless sensor and monitoring system for fall detection.

Many studies have performed motion analysis on daily activities and gait of various patients and healthy individuals [24,25,27-31], but not much research has focused on the causes and preventions for falling accidents from stairway at construction work sites. Moreover, development of wireless activity-monitoring sensors with real-time accident warning system for construction workers are rarely found in literature.

Therefore in this study, a wireless, lightweight, wearable textile insole pressure sensor for real-time safety monitoring system was developed to overcome and complement the limitations of commercial sensing systems. Performance and feasibility of the developed sensor were tested. Our sensor was designed to detect weight-shifting patterns and plantar pressure distribution changes during gait, ascending, and descending the stairs. Respective analysis of balance and postural changes during activities can be used to warn the workers when any extreme movements out of normal range of motion are detected.

2. Methods

2.1 Textile-based pressure sensor

A pressure sensor was developed by applying a parallel capacitance measurement method using conductive textile as shown in Fig. 1. For the sensor, W-290-PCN model (A-jin Electron, Gangseo, Busan, Republic of Korea) was used. The W-290-PCN model is consisted of polyester, sequentially-plated with nickel, copper, and nickel.

Fig. 1. Structure of a parallel capacitor

A capacitor is made of two conductive plates that are separated by a dielectric material. The plates accumulate electric charge; positive charge in one plate and negative charge in the other plate and it is measured in units of Farad (F). 1 F is the capacitance of 1 C charge stored at both ends of the conductor plate at a voltage of 1 V. Capacitance refers to the amount of charge per unit voltage a capacitor can store across a conductor plate and the parallel capacitance can be calculated by Equation (1).

\(C=\frac{Q}{V}=\varepsilon \frac{A}{d}\)       (1)

Where A means the area of plates, d means the distance between the two plates, and e means the permitivity of the material. Q refers to the electric charge and it is the amount of electricity a certain object has, and is divided into a positive charge and a negative charge. C is inversely proportional to the distance of the two plates and is proportional to the area of the material and the dielectric constant between two plates.

Two types of textile-based pressure sensors were developed; one for performance evaluation and the other for feasibility test during ascending/descending the stairs as illustrated in Fig. 2. Fig. 2 (a) shows the structure of a pressure sensor made for performance evaluation and it consists of two layers (a sensor layer and a ground layer). Fig. 2 (b) shows the structure of an insole-type pressure sensor. The sensor for performance evaluation was made with a single channel and the size of the insole was 5 x 5 cm². The insole type pressure sensor for feasibility test was built with 10 channels and its size was 270 mm as illustrated in Fig. 2 (c).

Fig. 2. Structure of the proposed pressure sensors, (a) performance test type, (b) insole type, (c) sensor location

2.2 Capacitance measurement and monitoring system

To measure the capacitance changes during gait, we develoepd a capacitance measurement system as illustrated in Fig. 3. Fig. 3 (a) shows the safety shoes integrated with capacitance measurement system and Fig. 3 (b) shows the structure of printed circuit board (PCB) of our proposed system. The size of PCB was 2.3 x 3.3 cm and the operation power was 3.7 V. For micro processor unit (MCU), STM32 series was used and for analog to digital signal conversion, MPR121QR2 sensor was used. Data were sampled at 100 Hz. We used Bluetooth communication to transfer data between the capacitance measurement board and C# capacitance moniotring system. The baudrate was set up at 115200 bps (bit per second).

Fig. 3. Capacitance measurement system, (a) safety shoes integrated with capacitance measurement system, (b) capacitance measurement PCB

Fig. 4 shows the capacitance monitoring system. It was developed using C# and was designed to receive data from all 20 channels of the insole sensors (10 from each foot). Capacitance monitoring system shows the waveform of the data collected in real time and each window shows the capacitance values from 10 channels (from each foot). The UI component includes a section with the list of port numbers and baudrate information, connect/disconnect button, and recording start/end (save) button. Data can be stored as a text file (100 samples per minute) and the raw data text file includes a time stamp (yy-dd-hh-mm-ss), where yy refers to the year, dd the day, hh the hour, mm the minute, and ss the second.

Fig. 4. Capacitance monitoring system

2.3 Signal processing

Data from each set of 10 channels measured from each side of the foot were added together to form one signal. Then, fast fourier transform (FFT) was performed using Matlab2015a to check the frequency component of the measured signal from the developed system and to confirm the frequency components to be distributed between 0 to 3 Hz as shown in Fig. 5. Pre-processing of the data was performed by applying the 4^th low-pass filter with a cut-off frequency of 3 Hz.

Fig. 5. Results of fast fourier transform analysis

We applied moving average filter (n = 5 samples) using Equation 2 to minimize the noise component of the data during stair ascending/descending, and then smoothing was performed (Fig. 6). After pre-processing, the high peaks were detected using the local maxima algorithm (Fig. 7). Local maxima algorithm is a heuristic algorithm based on hill climbing search and it picks out the highest value in the selected area. A peak is defined when there is no other higher value.

Fig. 6. Results of moving average filter on the data collected during stair descending, (a) raw data, (b) filtered data

Fig. 7. Results of peak detection using local maxima algorithm

\(y[i]=\frac{\sum_{j=1}^{M-5} x_{i}-M+j}{M}\)       (2)

Where, y[i] is the output and M is the number of used points in moving average.

2.4 Feature extraction

Data collected from the proposed sensor were analyzed by 10-fold cross-validation method, which identified stair ascending and descending activities. A total of 12 feafures were selected for analysis as illustrated in Fig. 8.

Fig. 8. Features for data analysis

The features analyzed by weka assessment tool were as follows : (1) the highest peak of heel strike from summation of 10 channels, (2) the highest peak of toe off from summation of 10 channels, (3) pressure difference between heel strike and toe off, (4) time intervals between heel strike and toe off, (5-8) raw data from channels 1, 2, 9, 10 at heel strike, and (9-12) raw data from channels 1, 2, 9, 10 at toe off.

2.5 Experimental protocol

The experimental protocol was composed of two sections. The first was performance evaluation of our textile pressure sensor and the second section was testing the feasibility of our sensor in identifying the difference between stair ascending and descending.

2.5.1 Resolution evaluation using load cell tensile compressor

Resolution of the proposed sensor was evaluated by three consecutive experiments using load cell tensile compressor; model MCT-1150 (A&D Company, Tokyo, Japan) as illustrated in Fig. 9. In order to measure the capacitance value in real time according to change of the distance and pressure count, the capacitance value was measured using the LCR meter (LCR-8110G, GWInstek, New Taipei City, Taiwan).

Fig. 9. Resolution test setting

(1) Test 1 : The textile pressure sensor was placed between two conductive textiles and the charge in capacitance was measured by narrowing the distance between the two plates every 0.1 mm.

(2) Test 2 : The same pressure was applied to the developed sensor for 5,000 times with a force of 46 N to evaluate the consistency of the measured values.

(3) Test 3 : Linearity of the collected data was examined with the same weight for 10 times from 1 N to 46 N, in increments of 1N (1 N is equal to 9.8 kg). The MCT-1150 model has a maximum range of 51 N, but 46 N was selected because when the plates get too close and touch each other, the data output becomes inaccurate. Our sensor was to be placed in insoles; considering that the pressure would rise when the sensor is worn inside the shoes, 46 N was considered the maximum force meaningful to be tested.

2.5.2 Subject characteristics

Five young, healthy male subjects were included in this preliminary study. Their average age was 28.40 (± 3.36 SD) years and the average BMI was 25.40 (± 2.24 SD). None of the participants had a medical history of lower extremity injury and pathologic symptoms including fracture, sprain, musculoskeletal disease, pain, neurological symptoms, muscle weakness, limited range of motion in the joints. All subjects received a 5-minute pre-training before participating in the experiment for familiarization of the experimental protocol and the experiment procedure was thoroughly explained for clear understanding. The experiment was conducted after obtaining consent from all subjects in writing. Subjects continuously ascended and descended a total of 22 stair cases in a comfortable speed each person chose.

Table 1. Subject Characteristics

2.5.3 Feasibility test for identifying stair ascending and descending

The morhologies of collected data were analyzed to test whether stair ascending and descending motions were identifiable as shown in Fig. 10. We used k-Nearest Neighborhood (k-NN) algorithm and differentiated stair ascending and descending movements using 10-fold cross-validation method of weka assessment tool. Normal walking data (20 m) were additionally collected as reference data.

Fig. 10. Experiment environment

3. Results

3.1 Results of sensor performance evaluation

The results of resolution performance test by using a load cell tensile compressor are shown in Table 2. As the distance between the two plates narrowed, the capacitance value was found to increase. The experiments confirmed that distance and capacitance were inversely proportional. As the distance increases by 1 mm, the capacitance value decreases by 0.07 pF in average (SD: 0.03 pF).

Table 2. Capacitance values at different distances between two plates

Trend equation for the capacitance value according to the distance changes was derived as illustrated in Fig. 11. The derived trend formula had R² equal to 0.99 and the fourth order polynomial y = 05E-06x⁴ + 0.0004x³ - 0.0057x² + 0.0803x + 1.3838.

Fig. 11. Results of capacitance and trend equation of the proposed sensor according to distance Consistency of the measured values while loading 46N of force for 5,000 times is shown in

Table 3. Average pressure was 2.4 pF with standard deviation of 0.06 pF.

Table 3. Consistency of the sensor capacitance values

Fig. 12 shows the results of linearity test after repeatedly loading the same pressure for 10times, and gradually increasing the pressure from 1 N to 46N in increments of 1N. The trend equation derived shows R² equaling 0.99 (y = 0.0216x + 1.3665)

Fig. 12. Linearity of capacitance value from 1 N to 46 N in increments of 1N of pressure

3.2 Results of feasibility test for identifying stair ascending and descending

Fig. 13 (a) presents the morphology of stair descending detected by the insole pressure sensor, where A (0.516 pF) was detected from the heel and B (0.516 pF) from the toe area. When descending the stairs, A and B peak points were found to be almost the same. Fig. 13 (b) shows the morphology of ascending the stairs. The peak value of B (0.518 pF) was higher r than that of A (0.514 pF). During stair ascending, subjects exerted more force on the toe area.

Fig. 13. Results of gait signal morphology during (a) descending, (b) ascending

Table 4 and 5 summarize the accuracy for ascending and descending the stairs using k-NN algorithm. The precision of ascending stairs was 0.872 and descending stairs was 0.909.

Table 4. Detailed accuracy for ascending and descending stairs

Table 5. onfusion matrix for ascending and descending stairs

4. Discussion

This study aimed to develop a textile-based insole pressure sensor (performance test type and feasibility test type) for real-time monitoring and warning system for the safety of construction workers. After sensor performance test, a pilot feasibility test was performed to examine whether the proposed sensor was able to identify ascending and descending motions on the stairs.

The results of our sensor performance test showed that at every distance increase in increments of 1 mm (from 7 mm to 4.5 mm), the capacitance value was decreased by 0.07 pF (SD : 0.03 pF). The highest capacitance of 3.40 pF was found in 4.5 mm thickness in our study. This could be used as a reference data for insole-type pressure sensor developers who are seeking or considering capacitive sensor for an alternative. In addition, the existing studies on the development of pressure sensors for gait measurement presented that more research is needed to find the optimum thickness of pressure sensing insoles. Our findings would provide a possible solution for developing more reliable insole-type textile pressure sensors. The results of the consistency test of our proposed sensor confirmed that our sensor has a high consistency (AVG : 2.4 pF, SD : 0.06 pF).

We chose the k-NN algorithm for feature analysis, because our proposed sensor was previously proved to have high resolution (less noise) when compoared with F-scan sensor [32]. The accuracy level of k-NN algorithm normally decreases when the feature variables are unrelated, have noise, or when the feature size matching is considered not important [33]. In this study, the precision of stair ascending and descending identified by k-NN algorithm confirmed high performance of our proposed sensor [34]. A study by Storm et al. (2015) reported high precision (99.2%) results of their sensor in differentiating ascending and descending stairs [35]. Their results were based on 8 different types of sensors worn over the entire body, which would be relatively more accurate in detecting the motions, but we tried using just one pair of insole sensors to reduce the cost for the future users and to make the wearing of the sensor more practical. Our results were confirmed to have as high performance compared to the previous study results (ascending 87.2%, descending 90.9%). Using less number of sensors is a key to reduce cost and increase work efficiency when the purpose is to be used in real life setting.

For the feasibility test of our sensor, weight shifting patterns during stair-climbing were analyzed based on the plantar pressure data. The results of capacitance values detected during stair descending showed no significant peak pressure difference between the heel and toe area. This was due to the entire foot being in contact with the surface of the stair case because the subjects were unconsciously trying to not fall off the stairs. Conversely during ascending, the force exerted on the toe area of the foot was larger than that of the heel area. Such morphology was found, because when climbing the stairs, the center of gravity of the human body is naturally directed forward and thus stronger force is applied to the forefoot area. This also is related to keeping the body from not falling. The results of this study were congruent with those of the previous studies that performed similar research [36].

Our sensor and monitoring system differentiated stair ascending from descending, and weight shifting changes during the activity were detected. This pilot test confirmed the feasibilty of the developed system to be used as fall prevention and danger warning system for construction worksites. Falling from the stairs is caused by momentary loss of balance, use of inappropriate weight shifting strategy, wrongful posture control during lifting or lowering of the body, and structural abnormality in the foot and lower extremity. These factors can be indirectly assessed and diagnosed by distribution and intensity changes in plantar pressure, therefore the results of this study are meaningful. Further studies with a larger number of subjects are needed, and testing various construction work-simulated activities would build meaningful standards for plantar pressure distribution and intensity. Moreover, individualized normal range of motion data can serve as a standard in detecting and alarming when any extreme movements out of normal range of motion is found.

Although we have developed a cost-effective, wearable textile pressure sensor and obtained meaningful results from the performance test and feasibility pilot, our research has a few limitations. First, the gait data are transmitted to the monitoring system via Bluetooth communications in real time, so the battery is consumed quickly. For this reason, data analyzing functions may not last for a long time, but battery life test was not performed in this study. Secondly, generalization of the results of this study may be difficult because the number of subjects was small. Lastly, the performance test was conducted in a laboratory and did not consider temperature and humidity factors.

In our future studies, we plan to recruit a larger number of subjects and apply our sensor in detecting various construction work-related movements and a real-life loading test can be designed based on the derived trend equation. As the results of this study showed feasibility, further study in various environments, considering changes in the temperature and humidity, at different heights and over uneven grounds could make better use of the developed sensor and monitoring system for construction workers.

5. Conclusion

We developed a light-weight, cost-effective, wearable insole-type textile pressure sensor with a Bluetooth-based monitoring system for predicting and alarming stairway falls of construction workers. The insole sensor was highly reliable and accurate in receiving and analyzing gait data, and could identify the difference between stair ascending and descending. Our proposed sensor and monitoring system are expected to detect more various activities in real time and prevent accidents by smart safety management of the construction sites.