Development of Plantar Pressure Measurement System and Personal Classification Study based on Plantar Pressure Image

Abstract

In this study, a Velostat pressure sensor was manufactured to develop a plantar pressure measurement system and a C#-based application was developed to monitor and collect plantar pressure data in real time. In order to evaluate the characteristics of the proposed plantar pressure measurement system, the accuracy of plantar pressure index and personal classification was verified by comparing with MatScan, a commercial plantar pressure measurement system. As a result, the output characteristics according to the weight of the Velostat pressure sensor were evaluated and a trend line with the reliability of r² = 0.98 was detected. The Root Mean Square Error(RMSE) of the weighted area was 11.315 cm², the RMSE of the x coordinate of Center of Pressure(CoP_x) was 1.036 cm and the RMSE of the y coordinate of Center of Pressure(CoP_y) was 0.936 cm. Finally, inaccuracy of personal classification, the proposed system was 99.47% and MatScan was 96.86%. Based on the advantage of being simple to implement and capable of manufacturing at low cost, it is considered that it can be applied to various fields of measuring vital signs such as sitting posture and breathing in addition to the plantar pressure measurement system.

1. Introduction

The pressure applied to the sole of the foot when the foot touches the ground during standing or walking is called plantar pressure (PP). As PP contains various information of the human body, the pressure data can be used to evaluate deformations of the foot, overall balance functions, and structural dysfunctions of the foot [1,2]. Recently, researches have been attempted to evaluate the accuracy of personal identification by extracting individual patterns from PP data [3,4].

Previously, behavioral characteristics of the human locomotion known as gait posture were extracted from gait video to be applied in the biometric analysis [5]. Biorecognition based on gait video analysis has been studied extensively because video recording systems are easy to install and are inexpensive. However, lowered accuracy of personal identification due to privacy invasion issues, environmental changes due to climate, and clothing can be a limitation [3]. On the other hand, using the PP pattern measured at the sole of the foot as a biorecognition technology has more advantages than video-recording the human body, because it causes relatively less privacy invasion and one’s PP pattern is difficult to imitate. A study by Todd C. Pataky et al. (2012) classified a total of 104 subjects based on dynamic PP data [4]. The authors reported that the subject classification rate was improved to more than 98% compared to classification rates from previous studies and personal identification based on PP pattern has a high potential to grow in the security and healthcare industries.

For PP pattern analysis, a sensitive, durable, light, and wearable accurate measuring system is required. Generally, the types of pressure-measuring sensors used for PP measurements are capacitive pressure sensors, piezoelectric pressure sensors, and resistor pressure sensors [6- 19]. Capacitive pressure sensor measures pressure by capacitance as the distance between two conductor plates changes according to the applied force. KF Lei et al. (2012) developed a capacitive pressure sensor for PP measurement using a polydimethylsiloxane (PDMS) material [9]. Capacitive pressure sensors have high accuracy and flexibility, but signal conditioning circuits and capacitance calculations are complex. Piezoelectric pressure sensors generate electric charges when force is applied from the outside, which can be used to measure the force applied to an object. Satu Karki et al. (2009) developed an insole-type pressure sensor using a piezoelectric polymer film for PP measurement [13]. Flexible piezoelectric materials can be used to measure pressure, but are sensitive to several other factors such as bending force.

Further research to develop an effective PP measurement system is important for the purpose of improving the PP measurement performance and system applicability [20]. And previous PP image-based classification researches mostly evaluated their classification accuracy by using the PP data collected from a commercial PP measurement system. In this study, a mat sensor was developed for PP measurement and a neural network model was developed for personal identification by generating PP images from the collected data.

2. Velostat-based PP measurement system

The plantar pressure measurement system proposed in this study is divided into a hardware part for measuring plantar pressure and data processing and communication, and a software part for monitoring and collecting plantar pressure data in real-time. In detail, it is divided into Velostat pressure sensor, plantar pressure data measurement circuit, 3D printing, and real-time monitoring application based on C# language.

The plantar pressure data measured by the Velostat pressure sensor is input to the Micro Controller Unit (MCU) through the Multiplexer chip and converted from analog data to digital data by the Analog to Digital Converter (ADC) chip. The MCU transmits the plantar pressure data converted into digital data to the C#-based monitoring application. The application displays and collects plantar pressure data in real-time. In order to remove the noise of the collected plantar pressure data, preprocessing including the median filter is performed, and then the plantar pressure index is derived and a plantar pressure image is generated.

2.1 Velostat pressure sensor

The Velostat pressure sensor (VPS) was manufactured in a 5-layer stacked structure as shown in Fig. 1. To measure the electrical resistance value of the Velostat, copper tapes were used as the input and output bus at the top and bottom of the Velostat. 35 pieces of 5x430 mm² copper tape were placed on the top, and 50 pieces of 5x295 mm² copper tape were placed at the bottom, perpendicular to the top copper tape. Double-sided tape with a width of 3 mm was placed between the copper tapes to prevent short circuits between the sensors. Finally, film paper, a non-conductive material, was placed to prevent noise generated when conductive materials such as the human body and external electricity contact with the copper tape that is used as input and output bus. The area where the pressure can be measured is the area where the copper tapes placed at the top and bottom of the Velostat intersect. The size of each area is 5x5 mm², and there are a total of 1,750 channels. The size of the manufactured VPS sensor is 295x430x0.4 mm³.

Fig. 1. Structure of Velostat pressure sensor

2.2 VPS measurement circuit

In this study, a voltage divider circuit was used to measure VPS with variable electrical resistance characteristics. The input voltage was 5 V, and the reference resistance was set to 100 Ω. Printed Circuit Board (PCB) was designed as in Fig. 2 to minimize the noise of the circuit measuring the VPS. In this study, AT91SAM3X8E was used as an MCU to measure the PP from the VPS. External power, power switch, power indicator, and Bluetooth are designed to increase the portability of the proposed system. FFC (Flexible Flat Cable) is designed to connect the circuit between the MCU and the multiplexer chip. The CD74HC406 Multiplexer chip was used to measure PP data from 50 output buses using 4 analog pins.

Fig. 2. PCB circuit for measuring VPS

The MCU sequentially measures a total of 1,750 channels and transmits the PP data to the application at a rate of 115,200 baud rate. The sampling frequency per channel is 2 Hz, which means that 1,750 channels of pressure data are transmitted twice per second.

2.3 3D printing case

3D printing was produced to protect the PCB from the outside and fix the VPS. The design of the hardware case is as shown in Fig. 3. The 3D printing case is composed of 7 modules. PCBs are located in A to F and the PCBs and the hardware case are combined using support. G is used to hold modules A through F. Fig. 3 (b) shows the appearance and overall size of all the module configurations of the hardware case connected.

Fig. 3. Design of 3D printing case

2.4 Monitoring application

Fig. 4 shows an application that can monitor and collect PP in real-time. The PP monitoring application consists of a port configuration part, data save part, and a PP monitoring part. In the port configuration part, UART (Universal Asynchronous Receiver/Transmitter) communication or Bluetooth communication is possible with the PP measurement sensor to be communicated. In the data save part, the PP RAW data is stored by linking the acquired time. Time information consists of seconds and milliseconds. In the PP monitoring part, the received PP data is reflected in real-time. The PP data were mapped to the jet color scale and expressed and a median filter was applied to remove noise.

Fig. 4. PP monitoring application

2.5 PP index calculation

In order to evaluate the accuracy of the proposed system compared to the commercial PP measurement system, the size of the weight-applied area and the location of the center of pressure (CoP) were calculated among various PP indicators. The size of the weight-applied area was selected to evaluate the accuracy of weight recognition for each channel, and the position of CoP was selected to evaluate the output accuracy for the weight of each channel. The size of the area to which the weight was applied and the location of CoP were calculated after removing the value smaller than the threshold from the median filtered PP data. The size of the area to which the weight was applied was multiplied by the channel area size after calculating the number of channels having a value greater than 0.

\(\operatorname{CoP}_{x}=\frac{\sum_{i=1}^{n} X_{i} \times P_{i}}{\sum_{i=1}^{n} P_{i}}, \operatorname{CoP}_{y}=\frac{\sum_{i=1}^{n} Y_{i} \times P_{i}}{\sum_{i=1}^{n} P_{i}}\)       (1)

The location of CoP was calculated based on equation (1) [7]. CoP_x is how much has been moved in the x-axis direction from the (0, 0) position and CoP_y is how much has been moved in the y-axis direction from the (0, 0) position. Index i is the channel of the pressure sensor, n is the total number of channels of the pressure sensor, P_i is the pressure value corresponding to the i channel, X_i is the position in the x-axis direction of the i channel and Y_i is the position in the y-axis direction of the i channel.

2.6 Neural network model for personal classification

For personal classification based on the PP image, a neural network model was developed using the Convolutional Neural Network (CNN) algorithm, which shows high performance in image classification. The structure of the neural network model proposed in this study is composed as shown in Table 1.

Table 1. Structure of neural network model for personal classification

The first convolution layer set the number of channels to 32, the kernel size to (3, 3), the activation function to ReLU, and zero padding so that the size of the output image and the size of the input image were the same. The ReLU outputs 0 when the input value is less than 0 and outputs the input value as it is when it is greater than 0. The ReLU activation function of the hidden layer can improve the learning speed of the neural network [21]. The number of channels in the second convolution layer is 64, and the rest were set the same as in the first convolution layer. The kernel size of the pooling layer was set to (2, 2) to reduce the size of the output image to 1/4 of the input image. The dropout layer randomly removed 50% of the number of nodes to prevent overfitting. In the flatten layer, the shape of the data is changed from 2D data to 1D data in order to deliver the extracted main features to the dense layer. In the first dense layer, the number of output nodes was set to 512 and the activation function was set to ReLU. In the last dense layer, the number of output nodes was set to 30, and the activation function was set to Softmax, which is mainly used for multi-class classification.

In the proposed neural network model, RMSProp was used for the optimization function and Categorical Cross Entropy was used for the loss function. The batch size was set to 32 and the number of epochs was set to 10. The model was trained and evaluated in the computer specifications shown in Table 2.

Table 2. computer specifications

For model training and evaluation, PP images generated from PP data collected from the proposed system and MatScan were used. For model train, validation and test, data for each class were classified as 70% train, 15% validation and 15% test data.

3. Experimental protocol

After dividing the 1,750 channel VPS into 25 areas as shown in Fig. 5, the average value of the channels marked with black squares for each area was used for performance evaluation. The task of classifying areas and selecting a representative channel in the area to use for characteristic evaluation reflects the characteristics of each area while shortening the work time required to perform the characteristic evaluation.

Fig. 5. 25 areas of the VPS

When evaluating the characteristics of the VPS, 3D printed weight guideline was used to place the weight in the center of the marked black square in Fig. 5. In the characteristic evaluation, the time interval between sampling and sampling was narrowed by setting the sampling frequency per channel to 50 Hz.

3.1 Evaluation of output characteristics by weight

At static PP, pressures from about 0 g to 2,000 g per 10x10 mm² occur [22]. Therefore, in this study, in order to evaluate the output characteristics according to the weight of the VPS, the weight was increased in increments of 200 g to 4,000 g at 200 g intervals in each area. After the weight was applied, the pressure data for 3 seconds has been saved and the average value was used for analysis. The same experimental protocol was performed in a total of 25 areas and data were measured continuously. Fig. 6 shows the experimental environment for evaluating the characteristics of the VPS.

Fig. 6. Experimental environment for characteristics evaluation

3.2 Output error and drift characteristic evaluation

To verify that an ideally constant pressure value is output when static weight is applied to the VPS, the output error and drift characteristics were evaluated. The experiment was conducted in the same environment as in Fig. 6. To evaluate the output error and drift characteristics, a weight of 4,000 g was applied to area 13, the middle area of the VPS, for 10 minutes 2 seconds. In order to remove the noise generated in the process of lifting and lowering the weight, the 1 second data immediately after the start of the experiment and just before the end of the experiment were removed. The output error was evaluated as the difference between the maximum value and the minimum value in the pressure data for 10 minutes, and the drift was evaluated as the difference between the average value for 2 seconds after the start of the experiment and the average value for 2 seconds before the end of the experiment [15].

\(R M S E(\text { Velostat, weight })=\sqrt{\frac{\sum_{i=1}^{n}\left(\text { Velostat }_{i}-\text { weight }_{i}\right)^{2}}{n}}\)      (2)

The output error of the VPS for the weight was calculated through Root Mean Square Error(RMSE) of equation (2). Velostati means the actual measured value corresponding to the index i and weighti mean the predicted value corresponding to the index i. In the experiment to evaluate the output error and drift characteristics, the weight was kept constant, so the weight corresponds to 1 kg.

3.3 PP measurement system performance evaluation

The experimental environment for measuring PP is shown in Fig. 7. MatScan, a commercial PP measurement system, was used as a reference system, and at the same time, the proposed system was placed on MatScan to collect PP data. Two PP measuring systems were fixed with tape so that the proposed system was not moved on the MatScan when the subject stood on the PP measuring system. The sampling frequency per channel of MatScan is set equal to the frequency of the proposed system as 2 Hz.

Fig. 7. Experimental environment for PP measurement

For the system proposed to monitor and collect PP data in real-time, a C#-based PP monitoring application was used, and for MatScan, the FootMat Research 7.10 program was used. In this experiment, physically healthy subjects (15 males and 15 females) were participated. When selecting subjects, subjects with a foot size that could exceed the measurement allowable range of the system proposed in this study and subjects who were difficult to stand for 1 minute were excluded from recruitment. Subjects with an average age of 24.77 ± 3.00 years participated in the experiment, and the average foot size, weight, and height were 253.83 ± 16.49 mm, 66.57 ± 12.34 kg and 167.33 ± 8.96 cm, respectively.

3.3.1 protocol

Table 3 shows the entire experimental protocol. Before the PP was measured, the purpose and method of the experiment were explained to the subject, and MatScan was calibrated. Upon successful completion of the MatScan calibration, the subject took off his shoes and stood on the PP measurement system with only socks on and maintained the standing position for 70 seconds. This was performed a total of 5 times, and the subject was instructed to come off the PP measurement system each time before measurement. In addition, the subject was instructed to minimize movement and not to talk for 70 seconds, and the posture of the foot was not restricted when standing. All experiments took approximately 590 seconds. In order to analyze the PP data collected from the proposed system and MatScan at the same time, the data for about 5 seconds immediately after the start of the experiment and just before the end of the experiment were removed. From the PP data collected from the proposed system and MatScan, the size of the weighted area and CoP_x and CoP_y were calculated, and to evaluate the accuracy of the PP index of the proposed system from the verified commercial PP measurement system, the RMSE was calculated based on equation (3). Velostati means the measured value at the index i of the proposed system, and MatScani means the measured value at the index i of the MatScan system.

Table 3. Experimental protocol for PP measurement

\(R M S E(\text { Velostat, MatScan })=\sqrt{\frac{\left.\sum_{i=1}^{n} \text { (Velostat }_{i}-\text { MatScan }_{i}\right)^{2}}{n}}\)      (3)

A PP image was generated from the PP data to evaluate the classification accuracy of the neural network model for personal classification. 600 PP images were generated per person, and a dataset was generated with the PP images of 30 participants in the experiment. A total of 18,000 PP images were used to build the classification model.

4. Results and Discussion

In order to verify the reliability of the PP measurement system proposed in this study, the characteristics of the VPS and the performance evaluation experiment of the proposed system compared to the commercial PP measurement system were conducted. Among the characteristics of the VPS, the output characteristics by weight, output error and drift characteristics were evaluated, and the performance evaluation of the proposed system evaluated the size of the weighted area and the accuracy of CoP position compared to the commercial PP measurement system. The size of the weighted area and the CoP position were verified through Bland-Altman. Analysis was carried out using the SPSS statics 25 program.

4.1 Result of VPS characteristic evaluation

Fig. 8 shows the average output curve of 25 areas. It can be seen that the output value increases as the weight increases. The trend line of the output voltage as a function of the applied weight is y = 1.143x³ -1.872x² + 1.009x + 0.011 and the reliability was high with r² = 0.983. x is the applied weight, y is the output voltage, and r² is the correlation between the trend line equation and the graph in Fig. 8. Fig. 9 shows the standard deviation of the output value for each area according to the weight. It can be seen that the standard deviation increases up to the weight of 200 g, but the standard deviation decreases from the subsequent weight. This means that from 250 g or more, the system output stabilizes as the weight increases.

Fig. 8. Output curve of VPS according to weight

Fig. 9. Standard deviation of VPS by area according to weight

Fig. 10 is a graph showing the output error and drift characteristics of the 13th area of VPS. The black line represents the raw signal and the red line represents the signal to which the moving average filter (Window size: 50) is applied. Among the data for 10 minutes, the maximum output was 1.155 kg and the minimum output was 1.120 kg, resulting in an output error of 0.035 kg. The average value of 100 samples for 2 seconds after the start of the experiment was 1.129 kg, and the average value of 100 samples for 2 seconds before the end of the experiment was 1.149 kg, resulting in a drift of 0.020 kg, and the RMSE of the VPS sensor for 1 kg was 0.138 kg.

Fig. 10. Error and drift characteristics of VPS

Fig. 11 shows the system response of the VPS when the weight is 4,000 g and it was confirmed that the system response was 80 ms.

Fig. 11. System response of VPS

4.2 Result of PP measurement system performance evaluation

In order to evaluate the accuracy of the PP index of the proposed PP measuring system compared to MatScan, a commercial PP measuring system, the RMSE for the size of the weighted area, CoP_x and CoP_y index were calculated. The RMSE of the weighted area was 11.315 cm², the RMSE of CoP_x was 1.036 cm and the RMSE of CoP_y was 0.936 cm. BlandAltman was used to evaluate the comparison between the proposed system and MatScan. Fig. 12 shows the Bland-Altman and scatter plot results of the size of the area to which the weight was applied, the CoP_x and CoP_y. Bland-Altman's y-axis represents the difference between the proposed system and MatScan and the x-axis represents the mean. The upper and lower limits were calculated based on the 95% (1.96 SD) confidence interval, and it can be seen that most of them except for some outliers were included in the 95% confidence interval.

Fig. 12. Results of Bland-Altman and scatter plot

4.3 Result of personal classification based on PP image

Table 4 is the result of train and validation of a neural network model for personal classification based on the PP image generated from the PP data acquired from the proposed system and MatScan. In this study, the validation performance was higher than the train performance. There are various reasons for thisresult, such as when regularization and dropout are applied only in train, the data characteristics disappear, or there may be samples that are easier to classify than the training data set in the validation data set [23]. In both Loss and Accuracy, it was confirmed that the proposed system was train and validation with higher performance than MatScan and the test of the proposed system was 99.47% and the test of MatScan was 96.86%. These results indicate that PP image-based individual classification is possible with the proposed system. In addition, although there is a slight difference, in the proposed environment, subjects were classified with an accuracy of 2.6% higher than that of a commercial PP measurement system.

Table 4. Results of personal classification based on PP images

In Fig. 13, y-axis on the left is loss, y-axis on the right is accuracy, and the x-axis is epoch. This is a visualization of the loss and accuracy results according to the epoch rotation of the proposed system and MatScan.

Fig. 13. Results of training and validation of neural network model for personal classification

Through the confusion matrix shown in Fig. 14, when 30 subjects are classified, correctly classified subjects and incorrectly classified subjects can be identified, and in the case of incorrectly classified subjects, it is possible to confirm which subject was incorrectly classified.

Fig. 14. Results of confusion matrix

5. Conclusion

In this study, in order to develop a PP measurement system, a VPS capable of measuring PP was manufactured and an application that can monitor and collect PP data in real time was developed. To verify the reliability of the proposed system, the characteristics evaluation of the VPS and the performance evaluation of the proposed PP measurement system were conducted. In a future study, we intend to improve the accuracy of the PP index by minimizing noise and errors in the size of each channel by fabricating VPS with a flexible PCB. In addition, we intend to develop a more practical plantar pressure measurement system by additionally analyzing PP indicators such as maximum pressure, average pressure, and ground reaction force.