Implementation of Falling Accident Monitoring and Prediction System using Real-time Integrated Sensing Data

Abstract

In 2015, the number of senior citizens aged 65 and over in Korea was 6,662,400, accounting for 13.1% of the total population. Along with these social phenomena, risk information related to the elderly is increasing every year. In particular, a fall accident caused by a fall can cause serious injury to an elderly person, so special attention is required. Therefore, in this paper, we implemented a system that monitors fall accidents and informs them in real time to minimize damage caused by falls. To this end, beacon-based indoor location positioning was performed and biometric information based on an integrated module was collected using various sensors. In other words, a multi-functional sensor integration module was designed based on Arduino to collect and monitor user's temperature, heart rate, and motion data in real time. Finally, through the analysis and prediction of measurement signals from the integrated module, damage from fall accidents can be reduced and rapid emergency treatment is possible. Through this, it is possible to reduce the damage caused by a fall accident, and rapid emergency treatment will be possible. In addition, it is expected to lead a new paradigm of safety systems through expansion and application to socially vulnerable groups.

1. Introduction

Through the Consumer Damage Surveillance System (CISS) for the past three years (2013- 2015), the number of cases of risk information related to the elderly aged 65 and over was 3,831 cases in 2013, 4,453 cases in 2014, and 5,327 cases in 2015. The number is increasing by about 900 every year[1-3]. In particular, a fall accident caused by a fall can cause serious injury to an elderly person, so special attention is required. According to the Centers for Disease Control and Prevention (CDC), the number one cause of injury among elderly people aged 65 years or older in the United States was falls, and in 2014, about 27,000 people died from falls. It is confirmed that 2.8 million people received emergency room treatment for a fall, and 800,000 of them had to be hospitalized[4,5]. In 2018, when looking at the mortality rate (per 100,000 population) of elderly people in their 60s or older in Korea, unintentional accidents were highest in the order of transportation accidents (24.7 people), fall accidents (17.2 people), and drowning accidents (2.8 people). Suicide (43.6 persons) is the highest among intentional accidents[6-8]. In particular, the death rate of accidents among the elderly is higher than that of other age groups.

Suicide accounts for the largest share in the deaths of the elderly. According to the 2018 ‘Social Survey’ of the National Statistical Office, 4.7% of the elderly over the age of 60 felt suicidal thoughts during the past year, and the reasons for suicidal thoughts were health problems (34.5%), economic difficulties (34.4%), and family discord (13.1%) and loneliness (12.9%). The OECD warns that the suicide rate among the elderly in Korea is 13 times that of Korea's top 10 suicides, and that the gap will increase[9-12]. Among the deaths caused by unintentional accidents of the elderly, transport accidents accounted for a very high proportion, followed by falls and fall-related deaths. Fall accidents on the same side and on the bed increase rapidly among the elderly in their 70s or older. In particular, deaths from falls in stairs, ladders, scaffoldings, buildings, etc. are increasing significantly in the 50s and older.

Fall and fall accidents require a lot of rehabilitation costs, and it is more important to prevent them in advance through the provision of correct information in that they are mostly preventable compared to other safety accidents. Like the elderly, the vulnerable class is exposed to a high risk of accidents. They are easily exposed to the risk of safety accidents due to their socioeconomic vulnerability as well as their physical or cognitive vulnerability[13]. Also, the risk of casualties in the event of an accident is much higher. Therefore, it is necessary to analyze the situation and characteristics of accidents and prepare differentiated safety policies and supports accordingly to ensure the safety of the elderly. In particular, since the absolute number of the elderly population and the ratio of the elderly to the population are on the rise, a safety policy taking this into account is required[14-16].

To solve this problem, strategic policy support, technology development, and application are needed for the safety management and service provision of socially vulnerable groups who are constantly exposed to new risks and neglected in the blind spot of safety accidents. Therefore, it is necessary to develop and build customized safety management services for socially vulnerable groups using ICT-based application services. ICT application services have been recently proposed as solutions for safety management and crime prevention because they are useful for real-time situation recognition, timely risk warning or response, and automatic identification or location identification. In such a social environment, safety services and system environments for the elderly are being built in recent years as they enter an ultraaging society[17-22].

Through the design of this study, it is possible to reduce the maintenance cost by requiring only one communication network line that can connect the base station, which is a relay service equipment, to the Internet. Beacon receivers and base stations can also be manufactured at low cost, so compared to GPS-based equipment, if Beacon is used, it is possible to manufacture at least 1/10. In addition, in the case of positioning, there is less distance measurement error than GPS, and it has the advantage of being advantageous for indoor positioning in terms of the performance of the beacon itself. It is possible to provide convenience to users for up to two years.

Therefore, in this paper, we implemented a system that monitors fall accidents and informs them in real time to minimize damage caused by falls. To this end, beacon-based indoor location positioning was performed and biometric information based on an integrated module was collected using various sensors. In other words, a multi-functional sensor integration module was designed based on Arduino to collect and monitor user's temperature, heart rate, and motion data in real time. The implementation procedure in this paper can be largely divided into two. The first is indoor positioning. To this end, a module was developed to analyze the efficiency of indoor location measurement using beacons by collecting and analyzing signals for indoor location measurement based on beacons. The second is biosignal measurement based on sensing information. That is, various sensors required for real-time situation recognition are designed as integrated modules, and a safety system module has been developed that analyzes the collected data and transmits processing results.

2. Positioning Module

In this paper, we developed a module to analyze indoor location measurement technology that can improve accuracy among various indoor location measurement techniques and is compatible with the underlying service. To this end, a module was designed and developed to analyze the efficiency of indoor location measurement using beacons by collecting and analyzing signals for indoor location measurement based on beacons used for indoor location measurement.

Fig. 1 shows the overall design overview for an indoor positioning integrated module utilizing beacons and ultrasonic sensors. It is designed to extend the life of the battery so that the beacon broadcasting by recognizing the spatial location information of the smartphone user can operate for a long time. In addition, a motion sensor was added to recognize the motion of a smartphone user's indoor spatial location[23-25]. In addition, an integrated module for recognizing the user's distance and ID by setting a specific coordinate indoors was designed to recognize the distance and user ID of the spatial location information. Smartphone users generate signals and use an app to track signals in order to be controlled by the operation server. At this time, the app also shares and installs a signal generator for generating a signal, and at the same time, it is set to a unique ID.

In order to more accurately locate the user's location in the indoor space, a number of signal sensors for recognizing information on facilities were installed at specific coordinates to collect information. At this time, the signal sensor recognizes the signal generated by the smartphone app, measures the distance of the signal, calculates it, and then identifies the spatial location and transmits it to the operation server[26,27].

In addition, a motion sensor to recognize moving objects in the smartphone's location space can be installed inside and outside the space. At this time, the motion sensor detects the spatial location and activates the broadcasting beacon to extend the life of the battery so that the beacon can operate for a long time. In addition, we designed a beacon that recognizes the motion transmitted from the motion sensor as well as the signal transmitted from the signal sensor, determines the spatial location, and then transmits it to the operation server.

Fig. 1. Positioning integration module configuration

A sensor node is a node that receives data from sensors generated from a dedicated wearable using a BLE communication module and transmits the detected data to a base station through BLE communication. Also, it measures the distance between wearable and Bluetooth communication, which is data used for positioning, and transmits it to the base station together with the data from the sensors. In addition, it is a data intermediate collection node that collects sensor node information using a mesh network and delivers the collected data to an analysis server through Internet communication. Because the base station is connected to multiple sensor nodes and an overload phenomenon occurs while collecting data, distributed processing is performed to prevent overload phenomenon in one base station by placing it in a certain area.

The analysis server analyzes and processes the data periodically received from the base station and provides the user information to the person in charge through the transmission to the smart equipment of the person in charge using Internet communication. In addition, the analysis server classifies emergency situations and normal situations through the measurement value analysis criteria, and in case of an emergency, the analysis data is delivered to the smart device of the person in charge immediately, and the analysis data is delivered to the smart device of the person in charge at regular intervals in case of an emergency.

Data is transmitted to the sensor node through BLE data communication between the beacon mounted on the wearable and the sensor node. The data for positioning is finally analyzed in the analysis server by measuring the distance between the wearable and adjacent sensor nodes. In addition, a motion sensor for recognizing a moving object in the location space of the smartphone can be installed inside and outside the space. At this time, the motion sensor detects the location of the space and operates the broadcast beacon to extend the life of the battery so that the beacon can operate for a long time. In addition, we designed a beacon that recognizes the motion transmitted from the motion sensor as well as the signal transmitted from the signal sensor to determine the spatial location and transmits it to the operation server.

The ultrasonic sensor consists of 5V, GND, Trig, and Echo terminals, the power supply is 800mA, and an integrated module was designed with iBeacon of the Bluetooth 4.0 Module NRF51822 Chipset. Fig. 2 shows the design of the integrated module with iBeacon of the Bluetooth 4.0 Module NRF51822 Chipset.

Fig. 2. Beacon-based integrated module design circuit diagram

3. Integration Sensor Module

In order to recognize the user's situation in real time, various sensors were designed into one integrated module and a multi-functional sensor integrated module was developed. The multifunctional sensor integration module developed in the paper was designed as an integrated module by linking the gyro sensor, temperature sensor, and heart rate sensor based on the ESP8266 NodeMCU process. Priority, NodeMCU, the main process, is a microcontroller designed by Espressif Systems and is an open source IoT platform, an MCU development board that implements WiFi functions. The ESP8266 itself is its own WiFi networking solution provided as a bridge from the existing microcontroller to WiFi. The module comes with a built-in USB connector and a variety of pin layouts, making sensor connection convenient[28,29]. Also, if you use a micro USB cable, you can connect the NodeMCU devkit to your laptop without any problems, just like Arduino.

To the next, the gyro sensor was designed in conjunction with the MPU-6050 module. The MPU-6050 motion processing unit is a motion processing solution with integrated 9-axis sensor fusion using a proprietary, field-proven MotionFusion™ engine for handset and tablet applications, game controllers, motion pointer remotes and other consumer devices. The MPU-6050 features a Digital Motion Processor™ (DMP™) hardware accelerator engine with auxiliary I2C ports that interface with digital sensors such as a 3-axis MEMS gyroscope, 3- axis MEMS accelerometer and magnetometer. When connected to a 3-axis magnetometer, the MPU-6050 provides 9-axis MotionFusion output to a basic I2C or SPI port. The MPU-6050 combines acceleration and rotation motion and title information into a single data stream for an application. This MotionProcessing™ technology integration has the advantage of providing a smaller footprint compared to discrete gyroscope and accelerometer solutions[30- 32].

Also, the non-contact type TMP007 was applied as a temperature sensor for measuring body temperature. The TMP007 is an infrared (IR) thermopile sensor that measures the temperature of an object without contacting it. The integrated thermopile measures the data in such a way that the sensor absorbs the infrared energy emitted from the object. The thermopile voltage is digitized and provided as input to the integrated math engine along with the die temperature (TDIE), which then calculates the object temperature. The TMP007 is compatible with I2C and SMBus interfaces and allows up to 8 devices on a single bus. The low power consumption combined with the low operating voltage makes it ideal for battery powered applications[33-35]. The TMP007 provides a convenient non-contact thermal solution for measuring temperature with factory-provided calibration, and is suitable for applications with custom system calibration.

Finally, SEN0203 was applied as the heart rate sensor. The SEN0203 Heart Rate Sensor is a thumb-sized heart rate monitor designed for Arduino microcontrollers. A Gravity interface is included for easy plug-and-play connectivity. The SEN0203 sensor is a pulse sensor developed based on PPG (Photo Plethysmo Graphy) technology, and is used as a simple and inexpensive optical technology that can be used to detect changes in blood volume in the microvascular layer of tissue[36]. Therefore, it is relatively easy to detect the pulsating component of the cardiac cycle according to this theory. Fig. 3 is the design of an integrated module that connects the gyro sensor, temperature sensor and heart rate sensor based on the NodeMCU main process.

Fig. 3. Integration module design schematic

The software module for data processing is configured in the Client/Server method. The client collects and transmits data from the device based on Arduino. In addition, the Server was developed to store and analyze data using Python. Fig. 4 shows the flow chart of the overall system where data collection and processing proceeds.

Fig. 4. Data processing system

In the Client Device software module, the WiFi communication method was applied. In addition, the initial control and measurement parameters of the gyro sensor were initialized. The normalized data and the amount of acceleration change were displayed by the calculation formula. In addition, sensor data such as temperature, heart rate, acceleration, and transmission were set and stored as the previous time and standby time to collect data per unit time. Fig. 5 shows a program for calculating acceleration in the gyro sensor.

The server software module checks the integrity of input data and receives device ID, temperature, heart rate, and motion data among the data. The received data is converted into a Python list type and returned. In addition, the received data was stored and outputted with the time and date of the input time. In data storage, errors of input data are checked, and if there are no errors, they are stored in the database. The server execution was developed so that data is input through UDP socket and stored after checking through process data.

Fig. 5. Acceleration calculation program

4. Classification Model Design

In deep learning classification modeling, a classification problem in which one of multiple answers is selected is called multi-classification. The activation function for multiclassification problems in deep learning networks mainly uses the softmax function. The softmax function is a method of estimating the probability for each class by receiving a kdimensional vector assuming that the total number of correct answers (classes) to be classified is k.

Softmax is an activation function that outputs a probability for each label, and the sum of all output values is 1. Softmax has the effect of emphasizing large values and weakening small ones. Assuming that the ith parameter in a k-dimensional vector is zi and the probability that the ith class is the correct answer is pi, the softmax function defines pi as follows. If there are three problems to be solved, since k=3, a 3D vector z=[z1 z2 z3] is input, and the softmax function returns the following output.

\(\begin{align}P_{i}=\frac{e^{z_{i}}}{\sum_{j=1}^{k} e^{z_{j}}} \quad(i=1,2,3, \ldots ., k)\end{align}\)      (1)

\(\begin{align}\operatorname{Sm}(z)=\left[\frac{e^{z_{1}}}{\sum_{j=1}^{3} e^{z_{j}}} \frac{e^{z_{2}}}{\sum_{j=1}^{3} e^{z_{j}}} \frac{e^{z_{3}}}{\sum_{j=1}^{3} e^{z_{j}}}\right]=\left[p_{1}, p_{2}, p_{3}\right]=\hat{y}\end{align}\)      (2)

At this time, \(\begin{align}\hat {y}\end{align}\) is the predicted value, and p1, p2, and p3 each represent the probability that class 1 is correct, the probability that class 2 is correct, and the probability that class 3 is correct, respectively, with a value between 0 and 1, with a total sum of becomes 1. That is, when there are k classes to be classified, a k-dimensional vector is received, the value of all vector parameters is changed to a value between 0 and 1, and a k-dimensional vector is returned again.

A representative loss function used in multiple classification is categorical_crossentropy. Entropy is a unit that quantifies data as a number[37]. Entropy is a value obtained by taking the logarithm of the reciprocal of the probability, and the reciprocal is taken because the higher the probability, the less important the data is. Crossentropy is similar to the expected value of each entropy multiplied by a probability. Instead of probability, the probability value predicted by the network is multiplied, and the value predicted by the network is called q(x). Binomials consist of from two categories, polynomials consist of three or more categories, and categorical cross entropy (CCE) means averaging the cross entropy of all categories.

\(\begin{align}C C E=\frac{1}{n} \sum_{i=1}^{n} p_{i}(x) \log \frac{1}{q_{i}(x)}\end{align}\)       (3)

Different from multiclass classification, a classification problem in which one of the two is chosen is called binomial classification. That is, when data with two labels comes in, it means to classify it as 0 or 1. In deep learning, binomial classification models mainly use the sigmoid function as an activation function. This is because the sigmoid function outputs either 0 or 1. It is also possible to use Softmax instead of Sigmoid. A representative algorithm used to solve such a binary classification problem is logistic regression. The hypothesis of logistic regression is not H(x)=Wx+b as in linear regression. We use the hypothesis of H(x)=f(Wx+b) by additionally using some specific function f that can create an S-shaped graph. A function f is already widely known and is called the sigmoid function. In other words, the function f for solving the binary classification problem, which is a hypothesis of logistic regression, is a sigmoid function[38].

The sigmoid function converges to 1 when the input value becomes infinitely large, and converges to 0 when the input value becomes infinitely small. The output of the sigmoid function ranges from 0 to 1, and is used for classification tasks using the above characteristics. If the threshold x (0 =< x =<1) is exceeded, it is classified as 1, and if it is not exceeded, it is classified as 0. A loss function typically used in binary classification is Cross Entropy Error (CEE).

H(x) = sigmoid(Wx + b)

= 1 + e - (Wx + b)

= σ(Wx + b)       (4)

\(\begin{align}C E E=-\sum_{i=1}^{n} t_{i} \log y_{i}\end{align}\)       (5)

t is the correct answer value, and y is the inference value. The number of correct answers and the number of inferences are the same. If this number is 2, it is binary classification, and if it is more than 2, it is multi-classification. The y value is a result value calculated by passing through several neural networks and becoming an input value of a special activation function at the end. Therefore, sigmoid is the most used activation function for binary classification, and Softmax is the most used for multiclass.

5. Results and Discussion

In this paper, the efficiency analysis for indoor location positioning through beacons and biosignal analysis using sensing data collected through the integrated module were performed. First, a distance measurement experiment was performed using a beacon-based indoor location positioning module. Data were collected at intervals of 1M, 2M, 3M, 5M and 10M for distance. As a result of the experiment according to the distance, slight differences occurred depending on the environment of the indoor space. More accurate measurement data could be collected within 3M. That is, in order to secure the reliability of positioning in an indoor space, it is determined that an environment in which a beacon module is installed at intervals within 3M to measure the position should be created. Fig. 6 shows the results of an experiment with a beacon module installed within 3M and indoor positioning. As can be seen from the experimental results, when the beacon module was installed within 3M, the user's location in the room could be positioned within the error range.

Fig. 6. Beacon module signal measurement result within 3M distance

As shown in Fig. 6, as a result of measuring the indoor location, the graphs were all positioned within 3M. A total of 6 beacons were installed in the same space, and the measured data for each beacon was stored at 0.5 second intervals. In Figure 5, the x-axis is time, which receives the object's position information at 0.5 second intervals and outputs it as a graph. Additionally, the y-axis displays the distance between the object and the beacon at 0.5 meter intervals. A specific object moves freely in the indoor space, and object location information is collected from 6 beacons. At this time, when the distance between the beacon and the object was within 3M, the most accurate location information could be obtained.

In addition, experiments according to the indoor space size and environment were also conducted. As a result of the experiment, the positioning of the beacon indoors was not related to the size of the indoor space. However, the error range was large depending on the environment such as the size and number of obstacles in the indoor space. In other words, there was almost no change in the measured value or a large or small indoor space. It can be seen that the user's indoor location can be immediately determined regardless of the space size. However, the experimental results of the medium-sized space showed a difference between the other two spaces and the distribution of data measurements. This is thought to be a hindrance to measuring the exact location of the user due to some obstacles installed in the intermediate space. As a result, positioning in an indoor space using beacons was able to extract more accurate measurement values in places where there are few obstacles in the space.

Also, a multi-functional sensor integrated module was designed and developed to recognize the real-time situation of the elderly fall accident, and the data was collected and analyzed through a SW module that processes the collected data in real time. The SW module for data processing is configured in a Client/Server method, and in the Client, data is collected and transmitted from the device based on Arduino. In addition, the server was developed to store and analyze data using Python. In the Client Device SW module, the WiFi communication method was applied, the initial control of the gyro sensor and measurement variables were initialized, and the normalized data and the acceleration change amount were displayed by the calculation formula.

In addition, sensor data such as temperature, heart rate, acceleration, and transmission are set and stored as the previous time and standby time to collect data per unit time. Server SW module checks the integrity of input data and receives Device ID, temperature, heart rate, and motion data among the data. The received data is converted into Python list type and returned. In addition, the inputted data was saved and printed with the time and date of the input point. In data storage, errors in the input data are checked and stored in the database only when there are no errors. In addition, the contents stored in the database are converted to Excel so that they can be used for analysis. Server execution was developed to receive data through a UDP socket, check through process data, and then save.

The collection and storage of data input through the sensor device were monitored and managed in real time in the server system. Fig. 7 shows the result of executing the processing SW module for the data collected through the multi-functional sensor integration module. The data processing SW module starts the server by running the Python server, checks the integrity of the data received through the multi-functional sensor integration module, and stores and outputs the data per second. In addition, the data stored in the database is converted to Excel so that the user's status information can be analyzed and stored.

Fig. 7. Biometric data measurement(left) and analysis data transformation(right)​​​​​​​

To the next, the data collected from the server system is transmitted to the manager system in real time, and in the manager monitoring mode, the transmitted data is visualized and expressed as a graph. Fig. 8 shows the monitoring screen in the manager system.

Fig. 8. Manager system monitoring screen​​​​​​​

Finally, for predicting fall accidents, deep learning analysis was performed using a binary classification model to determine the presence or absence of a fall accident based on the data measured through the sensor. The fall prediction data analysis process involves transmitting data using a wireless router once data is measured through the user's sensor device. In addition, the server system is configured to perform functions such as sensor data storage, data analysis, and alarm transmission for abnormal data. The collection and storage of data input through sensor devices can be monitored and managed in real time on a server system.

The input data used to predict fall accidents in this paper consists of three attributes and one class measured from body temperature, heart rate, and motion sensors, and consists of a total of 10,500 items. The three attribute information are stored by measuring the user's condition in real time, including body temperature, heart rate, and movement. The fourth and final piece of information was about falls and was expressed as 1 or 0. In other words, 1 corresponds to a case where a fall accident occurred, and 0 corresponds to a case where it did not.

To predict fall accidents, a model with one input layer, two hidden layers, and one output layer that predicts fall accidents was constructed. For the layers except the output layer, the most commonly used Relu function was used. Since many nodes were used, overfitting was prevented by arbitrarily turning off 20% of the nodes using dropout.

The error function used binary_crossentropy and Adam was used as the optimization function. The experiment was repeated until the entire sample was entered in 100 repetitions. Analysis was performed with 10 input values input at a time. As a result of testing with the test set, the accuracy was obtained as 0.8612. By applying the final prediction model, when the heart rate sensor measurement value was 31.5 or more and the motion sensor measurement value was 5.5 or more, the results derived from the fall accident were 86.12% consistent.

6. Conclusions

As we enter a super-aging society, the traditional family model is dismantling and family ties are weakening. In addition, social phenomena in which the bonds and affinity of local communities are deteriorating along with super-aging, urbanization, and single ethnicity are intensifying. As a result, the socially vulnerable groups are gradually increasing, and there is a limit to the management using human and material resources. Various methods have been proposed to solve these social problems. In particular, in the era of the 4th industrial revolution, safety management of the socially vulnerable class is being strategically promoted through the establishment of customized safety management systems and services using core technologies. Falls and falls are common injuries caused by falling or falling, but they can occur at any age. In particular, the incidence of falls in the elderly is increasing, leading to serious injury or complications from falls, leading to death. Falls and falls are the major causes of emergency room visits and hospitalizations due to injury, and are injuries with a large medical cost burden. The number of patients admitted to the emergency room due to falls and falls is reported to be 3 per 100 patients a year, and about 2 to 7 per 1,000 patients admitted to the hospital are known to be fall and fall patients.

Therefore, in this paper, we developed a system to increase safety for the elderly through real-time monitoring and notification service model implementation for fall accidents. To this end, a system was implemented that links the positioning of the elderly indoors and the collection and analysis of biometric information. Priority, a study was conducted to analyze indoor location measurement techniques that can improve accuracy among various indoor location measurement techniques and are compatible with the underlying service. To this end, signals for indoor location measurement were collected and analyzed based on beacons that are recently used for indoor location measurement in various fields. In addition, design and experiment were performed to analyze the efficiency of indoor location measurement using beacons, and the results were analyzed. As a result of the experiment, it was possible to secure the reliability of the measured data at a distance within 3M. In addition, in the experiment in the x-y coordinate space, it was possible to derive close results when the measured data in a space with small space size and almost no obstacles was compared with actual location information.

Also, a system capable of real-time situation recognition was developed to establish a safety management service environment for elderly fall accidents based on sensing information. To this end, a multi-functional sensor integrated module was designed based on Arduino to collect user's temperature, heart rate, and motion data in real time. In addition, by developing a realtime processing SW module for the collected data, a system for transmitting, processing, and analyzing the data input from the server was developed. As a result of the system simulation, the user's data is collected from the client device in real time, and transmitted to the server to output temperature data, heart rate data, and motion data per second, providing an environment for administrators to analyze.