• Proceedings of the KOR-BRONCHOESO Conference
• /
• 1981.05a
• /
• pp.7-8
• /
• 1981

Authors observed clinically 34 cases of the corrosive esophagitis caused by various corrosive agents at Kyung Hee University Hospital from Aug. 1978 to Dec. 1980. The results obtained were as follows; 1. Among the 34 patients, male was 19 (55.9%) and female 15(44.1%). Most frequently found age was 3rd decade. 2. 18 cases(52.9%) came to the hospital within 24 hours after ingestion of the agents, and 13 cases(38.2%) within 2 to 7 days. 3. Seasonal distribution showed most frequently in spring(35.3%). 4. The moment of the accident was suicidal attempt in 27 cases(79.4%) and misdrinking in 7 cases(20.6%). 5. Acetic acid was a most commonly used agent, showing 23 cases(67.6%), lye and insecticides were next in order. 6. Common chief complaints were swallowing difficulty and sore throat. 7. The average hospital days was 14.8 days. 8. Esophagogram was performed between 3 to 7 days after ingestion in 13 cases(38.2 %), findings were constrictions on the 1st narrowing portion in 4 cases(30.8%) and within normal limits in 3 cases(23.1%). 9. Esophagoscopy was performed in 31 cases(91.2%) between 2 to 7 days after ingestion, which revealed edema and coating on entrance of the esophagus in 9 cases (29.0 %). Diffuse edema on entire length of the esophagus and within normal limits were next in order. 10. Laboratory results were as follows: Anemia was in 1 cases(2.9%), leukocytosis. in 21 cases (61.8%), increase ESR in 9 cases (26.5%), markedly increased BUN and creatinine in 3 cases (8.8%), and hypokalemia in 1 cases(2.9%). Proteinuria in 10 cases(29.4%) hematuria in 4 cases(l1.8%), and coca cola urine in 3 cases (8.8%). 11. Associated diseases were 3 cases(8.8%) of cancer, 1 cases (2.9%) of diabetes mellitus, and 1 cases(2.9%) of manic depressive illness. 12. Various treatment was given: Esophageal and gastric washing in 23 cases(67.6%) for the emergent treatment, antibiotics in 32 cases(94.1%), steroids in 30 cases(88.2%), bougienation in 5 cases(14.7%), hemodialysis in 1 case(2.9%), and partial esophagectomy with gastrostomy and gastroileal anastomosis in 1 cases(2.9%). 13. Serious complications were observed in 9 cases (26.5%), consisted of 6 cases(17.6%) of esophageal stricture, 1 cases(2.9%), of aute renal failure, 1 cases (2.9%) of pneu momediastinum with pneumonia, and 1 cases (2.9%) of pneumonia.

• Proceedings of the Korean Institute of Intelligent Systems Conference
• /
• 1993.06a
• /
• pp.1041-1043
• /
• 1993

Fuzzy logic based Control Theory has gained much interest in the industrial world, thanks to its ability to formalize and solve in a very natural way many problems that are very difficult to quantify at an analytical level. This paper shows a solution for treating membership function inside hardware circuits. The proposed hardware structure optimizes the memoried size by using particular form of the vectorial representation. The process of memorizing fuzzy sets, i.e. their membership function, has always been one of the more problematic issues for the hardware implementation, due to the quite large memory space that is needed. To simplify such an implementation, it is commonly [1,2,8,9,10,11] used to limit the membership functions either to those having triangular or trapezoidal shape, or pre-definite shape. These kinds of functions are able to cover a large spectrum of applications with a limited usage of memory, since they can be memorized by specifying very few parameters ( ight, base, critical points, etc.). This however results in a loss of computational power due to computation on the medium points. A solution to this problem is obtained by discretizing the universe of discourse U, i.e. by fixing a finite number of points and memorizing the value of the membership functions on such points [3,10,14,15]. Such a solution provides a satisfying computational speed, a very high precision of definitions and gives the users the opportunity to choose membership functions of any shape. However, a significant memory waste can as well be registered. It is indeed possible that for each of the given fuzzy sets many elements of the universe of discourse have a membership value equal to zero. It has also been noticed that almost in all cases common points among fuzzy sets, i.e. points with non null membership values are very few. More specifically, in many applications, for each element u of U, there exists at most three fuzzy sets for which the membership value is ot null [3,5,6,7,12,13]. Our proposal is based on such hypotheses. Moreover, we use a technique that even though it does not restrict the shapes of membership functions, it reduces strongly the computational time for the membership values and optimizes the function memorization. In figure 1 it is represented a term set whose characteristics are common for fuzzy controllers and to which we will refer in the following. The above term set has a universe of discourse with 128 elements (so to have a good resolution), 8 fuzzy sets that describe the term set, 32 levels of discretization for the membership values. Clearly, the number of bits necessary for the given specifications are 5 for 32 truth levels, 3 for 8 membership functions and 7 for 128 levels of resolution. The memory depth is given by the dimension of the universe of the discourse (128 in our case) and it will be represented by the memory rows. The length of a world of memory is defined by: Length = nem (dm(m)＋dm(fm) Where: fm is the maximum number of non null values in every element of the universe of the discourse, dm(m) is the dimension of the values of the membership function m, dm(fm) is the dimension of the word to represent the index of the highest membership function. In our case then Length=24. The memory dimension is therefore 128*24 bits. If we had chosen to memorize all values of the membership functions we would have needed to memorize on each memory row the membership value of each element. Fuzzy sets word dimension is 8*5 bits. Therefore, the dimension of the memory would have been 128*40 bits. Coherently with our hypothesis, in fig. 1 each element of universe of the discourse has a non null membership value on at most three fuzzy sets. Focusing on the elements 32,64,96 of the universe of discourse, they will be memorized as follows: The computation of the rule weights is done by comparing those bits that represent the index of the membership function, with the word of the program memor . The output bus of the Program Memory (μCOD), is given as input a comparator (Combinatory Net). If the index is equal to the bus value then one of the non null weight derives from the rule and it is produced as output, otherwise the output is zero (fig. 2). It is clear, that the memory dimension of the antecedent is in this way reduced since only non null values are memorized. Moreover, the time performance of the system is equivalent to the performance of a system using vectorial memorization of all weights. The dimensioning of the word is influenced by some parameters of the input variable. The most important parameter is the maximum number membership functions (nfm) having a non null value in each element of the universe of discourse. From our study in the field of fuzzy system, we see that typically nfm 3 and there are at most 16 membership function. At any rate, such a value can be increased up to the physical dimensional limit of the antecedent memory. A less important role n the optimization process of the word dimension is played by the number of membership functions defined for each linguistic term. The table below shows the request word dimension as a function of such parameters and compares our proposed method with the method of vectorial memorization[10]. Summing up, the characteristics of our method are: Users are not restricted to membership functions with specific shapes. The number of the fuzzy sets and the resolution of the vertical axis have a very small influence in increasing memory space. Weight computations are done by combinatorial network and therefore the time performance of the system is equivalent to the one of the vectorial method. The number of non null membership values on any element of the universe of discourse is limited. Such a constraint is usually non very restrictive since many controllers obtain a good precision with only three non null weights. The method here briefly described has been adopted by our group in the design of an optimized version of the coprocessor described in [10].

• Journal of Intelligence and Information Systems
• /
• v.20 no.1
• /
• pp.195-211
• /
• 2014

Cloud computing services provide IT resources as services on demand. This is considered a key concept, which will lead a shift from an ownership-based paradigm to a new pay-for-use paradigm, which can reduce the fixed cost for IT resources, and improve flexibility and scalability. As IT services, cloud services have evolved from early similar computing concepts such as network computing, utility computing, server-based computing, and grid computing. So research into cloud computing is highly related to and combined with various relevant computing research areas. To seek promising research issues and topics in cloud computing, it is necessary to understand the research trends in cloud computing more comprehensively. In this study, we collect bibliographic information and citation information for cloud computing related research papers published in major international journals from 1994 to 2012, and analyzes macroscopic trends and network changes to citation relationships among papers and the co-occurrence relationships of key words by utilizing social network analysis measures. Through the analysis, we can identify the relationships and connections among research topics in cloud computing related areas, and highlight new potential research topics. In addition, we visualize dynamic changes of research topics relating to cloud computing using a proposed cloud computing "research trend map." A research trend map visualizes positions of research topics in two-dimensional space. Frequencies of key words (X-axis) and the rates of increase in the degree centrality of key words (Y-axis) are used as the two dimensions of the research trend map. Based on the values of the two dimensions, the two dimensional space of a research map is divided into four areas: maturation, growth, promising, and decline. An area with high keyword frequency, but low rates of increase of degree centrality is defined as a mature technology area; the area where both keyword frequency and the increase rate of degree centrality are high is defined as a growth technology area; the area where the keyword frequency is low, but the rate of increase in the degree centrality is high is defined as a promising technology area; and the area where both keyword frequency and the rate of degree centrality are low is defined as a declining technology area. Based on this method, cloud computing research trend maps make it possible to easily grasp the main research trends in cloud computing, and to explain the evolution of research topics. According to the results of an analysis of citation relationships, research papers on security, distributed processing, and optical networking for cloud computing are on the top based on the page-rank measure. From the analysis of key words in research papers, cloud computing and grid computing showed high centrality in 2009, and key words dealing with main elemental technologies such as data outsourcing, error detection methods, and infrastructure construction showed high centrality in 2010~2011. In 2012, security, virtualization, and resource management showed high centrality. Moreover, it was found that the interest in the technical issues of cloud computing increases gradually. From annual cloud computing research trend maps, it was verified that security is located in the promising area, virtualization has moved from the promising area to the growth area, and grid computing and distributed system has moved to the declining area. The study results indicate that distributed systems and grid computing received a lot of attention as similar computing paradigms in the early stage of cloud computing research. The early stage of cloud computing was a period focused on understanding and investigating cloud computing as an emergent technology, linking to relevant established computing concepts. After the early stage, security and virtualization technologies became main issues in cloud computing, which is reflected in the movement of security and virtualization technologies from the promising area to the growth area in the cloud computing research trend maps. Moreover, this study revealed that current research in cloud computing has rapidly transferred from a focus on technical issues to for a focus on application issues, such as SLAs (Service Level Agreements).

• Radiation Oncology Journal
• /
• v.19 no.3
• /
• pp.237-244
• /
• 2001

Purpose : To assess the tolerance, complete response rate, bladder preservation rate and survival rate in patients with muscle-invading bladder cancer treated with selective bladder preservation protocol. Method and Materials : From October 1990 to June 1998, twenty six patients with muscle-invading bladder cancer (clinical stage T2-4, N0-3, M0) were enrolled for the treatment protocol of bladder preservation. They were treated with maximal TURBT (transurethral resection of bladder tumor) and 2 cycles of MCV chemotherapy (methotrexate, crisplatin, and vinblastine) followed by $39.6\~45\;Gy$ pelvic irradiation with concomitant cisplatin. After complete urologic evaluation (biopsy or cytology), the patients who achieved complete response were planed for bladder preservation treatment and treated with consolidation cisplatin and radiotherapy (19.8 Gy). The patients who had incomplete response were planed to immediate radical cystectomy. If they refused radical cystectomy, they were treated either with TURBT followed by MCV or cisplatin chemotherapy and radiotherapy. The median follow-up duration is 49.5 months. Results : The Patients with stage T2-3a and T3b-4a underwent complete removal of tumor or gross tumor removal by TURBT, respectively. Twenty one out of 26 patients $(81\%)$ successfully completed the protocol of the planned chemo-radiotherapy. Seven patients had documented complete response. Six of them were treated with additional consolidation cisplatin and radiotherapy. One patient was treated with 2 cycles of MCV chemotherapy due to refusal of chemo-radiotherapy. Five of 7 complete responders had functioning tumor-free bladder. Fourteen patients of incomplete responders were further treated with one of the followings : radical cystectomy (1 patient), or TURBT and 2 cycles of MCV chemotherapy (3 patients), or cisplatin and radiotherapy (10 patients). Thirteen patients of them were not treated with planned radical cystectomy due to patients' refusal (9 patients) or underlying medical problems (4 patients). Among twenty one patients, 12 patients $(58\%)$ were alive with their preserved bladder, 8 patients died with the disease, 1 patient died of intercurrent disease. The 5 years actuarial survival rates according to CR and PR after MCV chemotherapy and cisplatin chemoradiotherapy were $80\%\;and\;14\%$, respectively (u=0.001). Conclusion : In selected patients with muscle-invading bladder cancer, the bladder preservation could be achieved by MCV chemotherapy and cisplatin chemo-radiotherapy. All patients tolerated well this bladder preservation protoco. The availability of complete TURBT and the responsibility of neoadjuvant chemotherapy and chemoradiotherapy were important predictors for bladder preservation and survival. The patients who had not achieved complete response after neoadjuvant chemotherapy and chemoradiotherapy should be immediate radical cystectomy. A randomized prospective trial might be essential to determine more accurate indications between cystectomy or bladder preservation.

• Radiation Oncology Journal
• /
• v.20 no.3
• /
• pp.221-227
• /
• 2002

Purpose : To evaluate the role of postoperative chemoradiotherapy in locally advanced rectal cancer, we retrospectively analyzed the treatment results of patients treated by curative surgical resection and postoperative chemoradiotherapy. Materials and Methods : From April 1989 through December 1998, 119 patients were treated with curative surgery and postoperative chemoradiotherapy for rectal carcinoma in Gyeongsang National University Hospital. Patient age ranged from 32 to 73 years, with a median age of 56 years. Low anterior resection was peformed in 59 patients, and abdominoperineal resection in 60. Forty-three patients were AJCC stage II and 76 were stage III. Radiation was delivered with 6 MV X rays using either AP-PA two fields, AP-PA both lateral four fields, or PA both lateral three fields. Total radiation dose ranged from 40 Gy to 56 Gy. In 73 patients, bolus infusions of 5-FU $(400\;mg/m^2)$ were given during the first and fourth weeks of radiotherapy. After completion of radiotherapy, an additional four to six cycles of 5-FU were given. Oral 5-FU (Furtulone) was given for nine months in 46 patients. Results : Forty $(33.7\%)$ of the 119 patients showed treatment failure. Local failure occurred in 16 $(13.5\%)$ patients, 1 $(2.3\%)$ of 43 stage II patients and 15 $(19.7\%)$ of 76 stage III patients. Distant failure occurred in 31 $(26.1\%)$ patients, among whom 5 $(11.6\%)$ were stage II and 26 $(34.2\%)$ were stage III. Five-year actuarial survival was $56.2\%$ overall, $71.1\%$ in stage II patients and $49.1\%$ in stage III patients (p=0.0008). Five-year disease free survival was $53.3\%$ overall, $68.1\%$ in stage II and $45.8\%$ in stage III (p=0.0006). Multivariate analysis showed that T stage and N stage were significant prognostic factors for five year survival, and that T stage, N stage, and preoperative CEA value were significant prognostic factors for five year disease free survival. Bowel complication occurred in 22 patients, and was treated surgically in 15 $(12.6\%)$, and conservatively in 7 $(5.9\%)$. Conclusion : Postoperative chemoradiotherapy was confirmed to be an effective modality for local control of rectal cancer, but the distant failure rate remained high. More effective modalities should be investigated to lower the distant failure rate.

• Radiation Oncology Journal
• /
• v.16 no.4
• /
• pp.367-375
• /
• 1998

Purpose : Xerostomia is a complication met by almost all patients who have radiotherapy for cancers of head and neck. Many studies for prevention of xerostomia will be necessary. Radiation-induced acute response of salivary glands has been defined as interphase death or apoptosis. Increased intracellular calcium level have an important role in radiation-induced apoptosis. Calcium channel blocker may prevent radiation-induced apoptosis of salivary glands. This study was designed to evaluate the effectiveness of diltiazem known as calcium channel blocker and pentoxifylline with inhibition of inflammatory response on the apoptosis as an acute response of radiation in rat salivary glands. Materials and Methods : Sprague-Dawley rats with about body weight 200-250 g were divided into 5 study groups : control, radiation alone, diltiazem with radiation, pentoxifylline with radiation, and diltiazem and pentoxifylline with radiation. The diltiazen and pentoxifylline were injected intraperitoneally 20 mg/kg and 50 mg/kg, 30 and 20 mimute before irradiation. respectively. Irradiation was given with a 4 MV linear accelerator. The 1600 cGy of radiation was delivered in a single fraction through a single anterior portal encompassing the entire neck. After 24 h of irradiation, rats were sacrificed and parotid and submandibular glands were removed and stained with hematoxylin and eosin. The quantification of apoptosis was performed by microscopic examination of stained tissue sections at a magnification of 200X and the percentage of apoptotic cell was calculated. Results : On parotid glands, the percentage of apoptosis by radiation alone, diltiazem with radiation, pentoxifylline with radiation, and diltiazem and pentoxifylline with radiation were 1.72$\%$ (8.35/486), 0.64$\%$ (2.9/453), 0.23$\%$ (1.2/516), and 0.28$\%$ (1.1/399), respectively. The apoptosis was markedly reduced in the groups receiving drugs compared with groups receivinge, radiation alone (p<0.05). In serous cell of submandibular glands, the percentages of apoptosis of radiation alone, diltiazem with radiation, pentoxifylline with radiation, and diltiazem and pentoxifylline with radiation were 1.94$\%$ (l1/567), 0.34$\%$ (1.9/554), 0.28$\%$ (1.8/637), and 0.22$\%$ (1.3/601), respectively. In the mucus cell of submandibular glands, the percentages of apoptosis were 0.92$\%$ (5.1/552), 0.41$\%$ (2.5/612), 0.29$\%$ (1.3/455), and 0.18$\%$ (1.0/562), respectively. The apoptosis was markedly reduced in the serous glands (p<0.05), but there was no difference in development of apoptosis in each group of mucus gland. Conclusion : These results suggest that radiation-induced apoptosis of serous cells of salivary glands may be decreased by diltiazem and pentoxifylline administration.

• Radiation Oncology Journal
• /
• v.20 no.3
• /
• pp.228-236
• /
• 2002

Purpose : Although high-dose-rate (HDR) brachytherapy regimens have been practiced with a variety of modalities and various degrees of success, few studies on the subject have been conducted. The purpose of this study was to compare the results of local control and late complication rate according to different HDR brachytherapy fractionation regimens in uterine cervical cancer patients. Methods and Materials : From November 1992 to March 1998, 224 patients with uterine conical cancer were treated with external beam irradiation and HDR brachytherapy. In external pelvic radiation therapy, the radiation dose was $45\~54\;Gy$ (median dose 54 Gy) with daily fraction size 1.8 Gy, five times per week. In HDR brachytherapy, 122 patients (Group A) were treated with three times weekly with 3 Gy to line-A (isodose line of 2 cm radius from source) and 102 patients (Group B) underwent the HDR brachytherapy twice weekly with 4 or 4.5 Gy to line-A after external beam irradiation. Iridium-192 was used as the source of HDR brachytherapy. Late complication was assessed from grade 1 to 5 using the RTOG morbidity grading system. Results : The local control rate (LCR) at 5 years was $80\%$ in group A and $84\%$ in group B (p=0.4523). In the patients treated with radiation therapy alone, LCR at 5 years was $60.9\%$ in group A and $76.9\%$ in group B (p=0.2557). In post-operative radiation therapy patients, LCR at 5 years was $92.6\%$ In group A and $91.6\%$ in group B (p=0.8867). The incidence of late complication was $18\%$ (22 patients) and $29.4\%$ (30 patients), of bladder complication was $9.8\%$ (12 patients) and $14.7\%$ (15 patients), and of rectal complication was $9.8\%$ (12 patients) and $21.6\%$ (22 patients), in group A and B, respectively. Lower fraction sized HDR brachytherapy was associated with decrease in late complication (p=0.0405) (rectal complication, p=0.0147; bladder complication, p=0.115). The same result was observed in postoperative radiation therapy patients (p=0.0860) and radiation only treated patients (0=0.0370). Conclusion : For radiation only treated patients, a greater number of itemized studies on the proper fraction size of HDR brachytherapy, with consideration for stages and prognostic factors, are required. In postoperative radiation therapy, the fraction size of HDR brachytherapy did not have much effect on local control, yet the incidence of late complication increased with the elevation in fraction size. We suggest that HDR brachytherapy three times weekly with 3 Gy could be an alternative method of therapy.

• Journal of Food Hygiene and Safety
• /
• v.21 no.4
• /
• pp.231-237
• /
• 2006

Nowadays the teabag is worldwide used for various products including green tea, tea, coffee, etc. since it is convenient for use. In case of outer packaging printed, however, there is a possibility that the plasticizers which is used for improvement in adhesiveness of printing ink may shift to inner tea bag. In this study, in order to monitor residual levels of plasticizers in teabags, we have established the simultaneous analysis method of 9 phthalates and 7 adipates plasticizers using gas chromatography (GC). These compounds were also confirmed using gas chromatography-mass spectrometry (GC-MSD). The recoveries of plasticizers analyzed by GC ranged from 82.7% to 104.6% with coefficient of variation of $0.6\sim2.7%$ and the correlation coefficients of each plasticizer was $0.9991\sim0.9999$. Therefore this simultaneous analysis method was showed excellent reproducibility and linearity. And limit of detection (LOD) and limit of quantitation (LOQ) on individual plasticizer were $0.1\sim3.5\;ppm\;and\;0.3\sim11.5\;ppm$ respectively. When 143 commercial products of teabag were monitored, no plasticizers analysed were detected in filter of teabag products. The migration into $95^{\circ}C$ water as food was also examined and the 16 plasticizers are not detected. In addition we carried out analysis of heavy metals, lead (Pb), cadmium (Cd), arsenic (As) and aluminum (Al) in teabag filters using ICP/AES. $Trace\sim23{\mu}g$ Pb per teabag and $0.6\sim1718{\mu}g$ Al per teabag were detected in materials of samples and Cd and As are detected less than LOQ (0.05 ppm). The migration levels of Pb and Al from teabag filter to $95^{\circ}C$ water were upto $11.5{\mu}g\;and\;20.8{\mu}g$ per teabag, respectively and Cd and As were not detected in exudate water of all samples. Collectively, these results suggest that there is no safety concern from using teabag filter.

• Journal of the Korean Society of Radiology
• /
• v.13 no.3
• /
• pp.371-379
• /
• 2019

The purpose of this study was to investigate the effect of the change in the X-ray condition on the entrance surface dose (ESD) and dose area product (DAP) in the cine imaging of coronary angiography (CAG), and to analyze the usefulness of the condition change on the dose relation and image quality by measuring and analyzing the Signal to Noise Radio (SNR) and Contrast to Nois Ratio (CNR) of the angiographic images taken by the Image J program. Data were collected from 33 patients (24 males and 9 females) who underwent CAG at this hospital from November 2017 to March 2018. In terms of imaging condition and data acquisition, the ESD and DAP of group A with a high tube current of 397.2 mA and group B with a low tube current of 370.7 mA were retrospectively obtained for comparison and analysis. For the SNR and CNR measurement and analysis via Image J, the result values were derived by substituting the obtained data into the formula. The correlations among ESD and DAP according to the change in the imaging condition, SNR, and CNR were analyzed by using the SPSS statistical analysis software. The relationships of groups A and B, having a difference in the imaging condition, mA, with ESD ($A:483.5{\pm}60.1$; $B: 464.4{\pm}39.9$) and DAP ($A:84.3{\pm}10.7$; $B:81.5{\pm}7$) were not statistically significant (p>0.05). In the relationships with SNR and CNR based on Image J, the SNR ($5.451{\pm}0.529$) and CNR ($0.411{\pm}0.0432$) of the images obtained via the left coronary artery (LCA) imaging of group B showed differences of $0.475{\pm}0.096$ and $-0.048{\pm}0.0$, respectively, from the SNR ($4.976{\pm}0.433$) and CNR ($0.459{\pm}0.0431$) of the LCA of group A. However, the differences were not statistically significant (p<0.05). In the SNR and CNR obtained via the right coronary artery (RCA) imaging, the SNR ($4.731{\pm}0.773$) and CNR ($0.354{\pm}0.083$) of group A showed increased values of $1.491{\pm}0.405$ and $0.188{\pm}0.005$, respectively, from the SNR ($3.24{\pm}0.368$) and CNR ($0.166{\pm}0.033$) of group B. Among these, CNR was statistically significant (p<0.05). In the correlation analysis, statistically significant results were shown in SNR (LCA) and CNR (LCA); SNR (RCA) and CNR (RCA); ESD and DAP; ESD and sec; DAP and CNR (RCA); and DAP and sec (p<0.05). As a result of the analyses on the image quality evaluation and usefulness of the dose change, the SNR and CNR were increased in the RCA images of the CAG obtained by increasing the mA. Based on the result that CNR showed a statistically significant difference, it is believed that the contrast in the image quality can be further improved by increasing the mA in RCA imaging.

• Journal of Intelligence and Information Systems
• /
• v.25 no.1
• /
• pp.163-177
• /
• 2019

As smartphones are getting widely used, human activity recognition (HAR) tasks for recognizing personal activities of smartphone users with multimodal data have been actively studied recently. The research area is expanding from the recognition of the simple body movement of an individual user to the recognition of low-level behavior and high-level behavior. However, HAR tasks for recognizing interaction behavior with other people, such as whether the user is accompanying or communicating with someone else, have gotten less attention so far. And previous research for recognizing interaction behavior has usually depended on audio, Bluetooth, and Wi-Fi sensors, which are vulnerable to privacy issues and require much time to collect enough data. Whereas physical sensors including accelerometer, magnetic field and gyroscope sensors are less vulnerable to privacy issues and can collect a large amount of data within a short time. In this paper, a method for detecting accompanying status based on deep learning model by only using multimodal physical sensor data, such as an accelerometer, magnetic field and gyroscope, was proposed. The accompanying status was defined as a redefinition of a part of the user interaction behavior, including whether the user is accompanying with an acquaintance at a close distance and the user is actively communicating with the acquaintance. A framework based on convolutional neural networks (CNN) and long short-term memory (LSTM) recurrent networks for classifying accompanying and conversation was proposed. First, a data preprocessing method which consists of time synchronization of multimodal data from different physical sensors, data normalization and sequence data generation was introduced. We applied the nearest interpolation to synchronize the time of collected data from different sensors. Normalization was performed for each x, y, z axis value of the sensor data, and the sequence data was generated according to the sliding window method. Then, the sequence data became the input for CNN, where feature maps representing local dependencies of the original sequence are extracted. The CNN consisted of 3 convolutional layers and did not have a pooling layer to maintain the temporal information of the sequence data. Next, LSTM recurrent networks received the feature maps, learned long-term dependencies from them and extracted features. The LSTM recurrent networks consisted of two layers, each with 128 cells. Finally, the extracted features were used for classification by softmax classifier. The loss function of the model was cross entropy function and the weights of the model were randomly initialized on a normal distribution with an average of 0 and a standard deviation of 0.1. The model was trained using adaptive moment estimation (ADAM) optimization algorithm and the mini batch size was set to 128. We applied dropout to input values of the LSTM recurrent networks to prevent overfitting. The initial learning rate was set to 0.001, and it decreased exponentially by 0.99 at the end of each epoch training. An Android smartphone application was developed and released to collect data. We collected smartphone data for a total of 18 subjects. Using the data, the model classified accompanying and conversation by 98.74% and 98.83% accuracy each. Both the F1 score and accuracy of the model were higher than the F1 score and accuracy of the majority vote classifier, support vector machine, and deep recurrent neural network. In the future research, we will focus on more rigorous multimodal sensor data synchronization methods that minimize the time stamp differences. In addition, we will further study transfer learning method that enables transfer of trained models tailored to the training data to the evaluation data that follows a different distribution. It is expected that a model capable of exhibiting robust recognition performance against changes in data that is not considered in the model learning stage will be obtained.