Video Analysis System for Action and Emotion Detection by Object with Hierarchical Clustering based Re-ID (계층적 군집화 기반 Re-ID를 활용한 객체별 행동 및 표정 검출용 영상 분석 시스템)
-
- Journal of Intelligence and Information Systems
- /
- v.28 no.1
- /
- pp.89-106
- /
- 2022
Recently, the amount of video data collected from smartphones, CCTVs, black boxes, and high-definition cameras has increased rapidly. According to the increasing video data, the requirements for analysis and utilization are increasing. Due to the lack of skilled manpower to analyze videos in many industries, machine learning and artificial intelligence are actively used to assist manpower. In this situation, the demand for various computer vision technologies such as object detection and tracking, action detection, emotion detection, and Re-ID also increased rapidly. However, the object detection and tracking technology has many difficulties that degrade performance, such as re-appearance after the object's departure from the video recording location, and occlusion. Accordingly, action and emotion detection models based on object detection and tracking models also have difficulties in extracting data for each object. In addition, deep learning architectures consist of various models suffer from performance degradation due to bottlenects and lack of optimization. In this study, we propose an video analysis system consists of YOLOv5 based DeepSORT object tracking model, SlowFast based action recognition model, Torchreid based Re-ID model, and AWS Rekognition which is emotion recognition service. Proposed model uses single-linkage hierarchical clustering based Re-ID and some processing method which maximize hardware throughput. It has higher accuracy than the performance of the re-identification model using simple metrics, near real-time processing performance, and prevents tracking failure due to object departure and re-emergence, occlusion, etc. By continuously linking the action and facial emotion detection results of each object to the same object, it is possible to efficiently analyze videos. The re-identification model extracts a feature vector from the bounding box of object image detected by the object tracking model for each frame, and applies the single-linkage hierarchical clustering from the past frame using the extracted feature vectors to identify the same object that failed to track. Through the above process, it is possible to re-track the same object that has failed to tracking in the case of re-appearance or occlusion after leaving the video location. As a result, action and facial emotion detection results of the newly recognized object due to the tracking fails can be linked to those of the object that appeared in the past. On the other hand, as a way to improve processing performance, we introduce Bounding Box Queue by Object and Feature Queue method that can reduce RAM memory requirements while maximizing GPU memory throughput. Also we introduce the IoF(Intersection over Face) algorithm that allows facial emotion recognized through AWS Rekognition to be linked with object tracking information. The academic significance of this study is that the two-stage re-identification model can have real-time performance even in a high-cost environment that performs action and facial emotion detection according to processing techniques without reducing the accuracy by using simple metrics to achieve real-time performance. The practical implication of this study is that in various industrial fields that require action and facial emotion detection but have many difficulties due to the fails in object tracking can analyze videos effectively through proposed model. Proposed model which has high accuracy of retrace and processing performance can be used in various fields such as intelligent monitoring, observation services and behavioral or psychological analysis services where the integration of tracking information and extracted metadata creates greate industrial and business value. In the future, in order to measure the object tracking performance more precisely, there is a need to conduct an experiment using the MOT Challenge dataset, which is data used by many international conferences. We will investigate the problem that the IoF algorithm cannot solve to develop an additional complementary algorithm. In addition, we plan to conduct additional research to apply this model to various fields' dataset related to intelligent video analysis.
The East Nine Royal Tombs is a representative place in the Royal Tombs of Joseon (a World Heritage Site). It consists of 1,289 stone artifacts including 979 related stone structures, 310 stone statues, and objects. Most of the stone structures in the East Nine Royal Tombs are composed of biotite granite, but some tombs are composed of light red granite. As a result of magnetic susceptibility measurement, the average data from Geonwolleung to Mongneung, excluding Hyeolleung, were similar, so it is estimated that stones were obtained from the same quarry. In the case of Sungneung, Sureung, and Gyeongneung, the range of susceptibility measurement is widely distributed. It assumed that the newly produced stones were mixed in the moving and construction process. Also, stones might be gathered from different quarries. As a result of a conservation status investigation, both the mound member and the ridge stone had the highest damage rate due to peeling and granular decomposition according to surface weathering. In the case of surface discoloration, yellowing and soils were found in the burial mound members. Yellowing, blackening, and soil were identified in the ridge stone structures. Bio-degradation is the major factor of deterioration of the East Nine Royal Tombs and the conservation status of the tombs were detected as grades 4 to 5. It seems that it is easy for the environment of the royal tombs to form soil for the microorganisms and fine conditions for continuous moisture. In the case of structures, they are in relatively good condition. As a result of a comprehensive damage rating for each tomb, the overall condition is good, but the Geonwolleung Royal Tomb and Hyeolleung Tomb, which were created in the early period, had relatively high weathering ratings. Stone objects in East Nine Royal Tombs have lost many pieces and gateway members due to surface deterioration. Also, secondary damage is ongoing. Each damage factor of the stone artifacts of the East Nine Royal Tombs combines to cause various and continuous damages. Therefore, it is necessary to establish regular conservation status data of the stone artifacts for efficient management after processing as well as conservation treatment of the royal tombs, and specific management manuals and systems. This study investigated the conservation status of stone structures in the East Nine Royal Tombs, a World Heritage Site, and systematically classified them to provide priority and necessity for conservation processing. We look forward to establishing a plan for the conservation and management of the East Nine Royal Tombs with this database in the future.
Recently, maintenance engineering and technology for civil and building structures have begun to draw big attention and actually the number of structures that need to be evaluate on structural safety due to deterioration and performance degradation of structures are rapidly increasing. When stiffness is decreased because of deterioration of structures and member cracks, dynamic characteristics of structures would be changed. And it is important that the damaged areas and extent of the damage are correctly evaluated by analyzing dynamic characteristics from the actual behavior of a structure. In general, typical measurement instruments used for structure monitoring are dynamic instruments. Existing dynamic instruments are not easy to obtain reliable data when the cable connecting measurement sensors and device is long, and have uneconomical for 1 to 1 connection process between each sensor and instrument. Therefore, a method without attaching sensors to measure vibration at a long range is required. The representative applicable non-contact methods to measure the vibration of structures are laser doppler effect, a method using GPS, and image processing technique. The method using laser doppler effect shows relatively high accuracy but uneconomical while the method using GPS requires expensive equipment, and has its signal's own error and limited speed of sampling rate. But the method using image signal is simple and economical, and is proper to get vibration of inaccessible structures and dynamic characteristics. Image signals of camera instead of sensors had been recently used by many researchers. But the existing method, which records a point of a target attached on a structure and then measures vibration using image processing technique, could have relatively the limited objects of measurement. Therefore, this study conducted shaking table test and field load test to verify the validity of the method that can measure multi-point displacement responses of structures using image processing technique.
In this study, the mechanical stability of red mud was improved for its commercial use as a catalyst to effectively decompose HFC-134a, one of the seven major greenhouse gases. Red mud is an industrial waste discharged from aluminum production, but it can be used for the decomposition of HFC-134a. Red mud can be manufactured into a catalyst via the crushing-preparative-compression molding-firing process, and it is possible to improve the catalyst performance and secure mechanical stability through calcination. In order to determine the optimal heat treatment conditions, pellet-shaped compressed red mud samples were calcined at 300, 600, 800 ℃ using a muffle furnace for 5 hours. The mechanical stability was confirmed by the weight loss rate before and after ultra-sonication after the catalyst was immersed in distilled water. The catalyst calcined at 800 ℃ (RM 800) was found to have the best mechanical stability as well as the most catalytic activity. The catalyst performance and durability tests that were performed for 100 hours using the RM 800 catalyst showed thatmore than 99% of 1 mol% HFC-134a was degraded at 650 ℃, and no degradation in catalytic activity was observed. XRD analysis showed tri-calcium aluminate and gehlenite crystalline phases, which enhance mechanical strength and catalytic activity due to the interaction of Ca, Si, and Al after heat treatment at 800 ℃. SEM/EDS analysis of the durability tested catalysts showed no losses in active substances or shape changes due to HFC-134a abasement. Through this research, it is expected that red mud can be commercialized as a catalyst for waste refrigerant treatment due to its high economic feasibility, high decomposition efficiency and mechanical stability.
The advent of digital media has facilitated easy access for adolescents to environments conducive to the purchase of narcotics. In particular, there's an increasing trend in the purchase and consumption of narcotics mediated through Social Network Services (SNS) and messenger services. Adolescents, sensitive to such environments, are at risk of experiencing neurological and mental health issues due to narcotic addiction, increasing their exposure to criminal activities, hence necessitating national-level management and support. Consequently, the quest for sustainable treatment methods for adolescents exposed to narcotics emerges as a critical challenge. In the context of high relapse rates in narcotic addiction, the necessity for cost-effective and user-friendly treatment programs is emphasized. This study conducts a literature review aimed at utilizing digital platforms to create an environment where adolescents can voluntarily participate, focusing on the development of therapeutic content through art. Specifically, it reviews societal perceptions and treatment statuses of adolescent drug addiction, analyzes the impact of narcotic addiction on adolescent brain activity and cognitive function degradation, and explores approaches for developing digital therapeutics to promote the rehabilitation of the addicted brain through analysis of precedential case studies. Moreover, the study investigates the benefits that the integration of digital therapeutic approaches and art therapy can provide in the treatment process and proposes the possibility of enhancing therapeutic effects through various treatment programs such as drama therapy, music therapy, and art therapy. The application of art therapy methods is anticipated to offer positive effects in terms of tool expansion, diversification of expression, data acquisition, and motivation. Through such approaches, an enhancement in the effectiveness of treatments for adolescent narcotic addiction is anticipated. Overall, this study undertakes foundational research for the development of digital therapeutics and related applications, offering economically viable and sustainable treatment options in consideration of the societal context of adolescent narcotic addiction.
We investigated and estimated at the characteristics of decomposition and by-products of N-Nitrosodimethylamine (NDMA) using a design of experiment (DOE) based on the Box-Behken design in an UV process, and also the main factors (variables) with UV intensity(
In this study, formation background of biodiversity and its changes in the process of geologic history, and effects of climate change on biodiversity and human were discussed and the alternatives to reduce the effects of climate change were suggested. Biodiversity is 'the variety of life' and refers collectively to variation at all levels of biological organization. That is, biodiversity encompasses the genes, species and ecosystems and their interactions. It provides the basis for ecosystems and the services on which all people fundamentally depend. Nevertheless, today, biodiversity is increasingly threatened, usually as the result of human activity. Diverse organisms on earth, which are estimated as 10 to 30 million species, are the result of adaptation and evolution to various environments through long history of four billion years since the birth of life. Countlessly many organisms composing biodiversity have specific characteristics, respectively and are interrelated with each other through diverse relationship. Environment of the earth, on which we live, has also created for long years through extensive relationship and interaction of those organisms. We mankind also live through interrelationship with the other organisms as an organism. The man cannot lives without the other organisms around him. Even though so, human beings accelerate mean extinction rate about 1,000 times compared with that of the past for recent several years. We have to conserve biodiversity for plentiful life of our future generation and are responsible for sustainable use of biodiversity. Korea has achieved faster economic growth than any other countries in the world. On the other hand, Korea had hold originally rich biodiversity as it is not only a peninsula country stretched lengthily from north to south but also three sides are surrounded by sea. But they disappeared increasingly in the process of fast economic growth. Korean people have created specific Korean culture by coexistence with nature through a long history of agriculture, forestry, and fishery. But in recent years, the relationship between Korean and nature became far in the processes of introduction of western culture and development of science and technology and specific natural feature born from harmonious combination between nature and culture disappears more and more. Population of Korea is expected to be reduced as contrasted with world population growing continuously. At this time, we need to restore biodiversity damaged in the processes of rapid population growth and economic development in concert with recovery of natural ecosystem due to population decrease. There were grand extinction events of five times since the birth of life on the earth. Modern extinction is very rapid and human activity is major causal factor. In these respects, it is distinguished from the past one. Climate change is real. Biodiversity is very vulnerable to climate change. If organisms did not find a survival method such as 'adaptation through evolution', 'movement to the other place where they can exist', and so on in the changed environment, they would extinct. In this respect, if climate change is continued, biodiversity should be damaged greatly. Furthermore, climate change would also influence on human life and socio-economic environment through change of biodiversity. Therefore, we need to grasp the effects that climate change influences on biodiversity more actively and further to prepare the alternatives to reduce the damage. Change of phenology, change of distribution range including vegetation shift, disharmony of interaction among organisms, reduction of reproduction and growth rates due to odd food chain, degradation of coral reef, and so on are emerged as the effects of climate change on biodiversity. Expansion of infectious disease, reduction of food production, change of cultivation range of crops, change of fishing ground and time, and so on appear as the effects on human. To solve climate change problem, first of all, we need to mitigate climate change by reducing discharge of warming gases. But even though we now stop discharge of warming gases, climate change is expected to be continued for the time being. In this respect, preparing adaptive strategy of climate change can be more realistic. Continuous monitoring to observe the effects of climate change on biodiversity and establishment of monitoring system have to be preceded over all others. Insurance of diverse ecological spaces where biodiversity can establish, assisted migration, and establishment of horizontal network from south to north and vertical one from lowland to upland ecological networks could be recommended as the alternatives to aid adaptation of biodiversity to the changing climate.
Backgrounds: The injury of a tissue results in the infalmmation, and the imflammed tissue is replaced by the normal parenchymal cells during the process of repair. But, constitutional or repetitive damage of a tissue causes the deposition of collagen resulting in the loss of its function. These lesions are found in the lung of patients with idiopathic pulmonary fibrosis, complicated fibrosis after diffuse alveolar damage (DAD) and inorganic dust-induced lung fibrosis. The tissue from lungs of patients undergoing episodes of active and/or end-stage pulmonary fibrosis shows the accumulation of inflammatory cells, such as mononuclear cells, neutrophils, mast cells and eosinophils, and fibroblast hyperplasia. In this regard, it appears that the inflammation triggers fibroblast activation and proliferation with enhanced matrix synthesis, stimulated by inflammatory mediators such as interleukin-1 (IL-1) and/or tumor necrosis factor (TNF). It has been well known that TGF-
The wall shear stress in the vicinity of end-to end anastomoses under steady flow conditions was measured using a flush-mounted hot-film anemometer(FMHFA) probe. The experimental measurements were in good agreement with numerical results except in flow with low Reynolds numbers. The wall shear stress increased proximal to the anastomosis in flow from the Penrose tubing (simulating an artery) to the PTFE: graft. In flow from the PTFE graft to the Penrose tubing, low wall shear stress was observed distal to the anastomosis. Abnormal distributions of wall shear stress in the vicinity of the anastomosis, resulting from the compliance mismatch between the graft and the host artery, might be an important factor of ANFH formation and the graft failure. The present study suggests a correlation between regions of the low wall shear stress and the development of anastomotic neointimal fibrous hyperplasia(ANPH) in end-to-end anastomoses. 30523 T00401030523 ^x Air pressure decay(APD) rate and ultrafiltration rate(UFR) tests were performed on new and saline rinsed dialyzers as well as those roused in patients several times. C-DAK 4000 (Cordis Dow) and CF IS-11 (Baxter Travenol) reused dialyzers obtained from the dialysis clinic were used in the present study. The new dialyzers exhibited a relatively flat APD, whereas saline rinsed and reused dialyzers showed considerable amount of decay. C-DAH dialyzers had a larger APD(11.70
The wall shear stress in the vicinity of end-to end anastomoses under steady flow conditions was measured using a flush-mounted hot-film anemometer(FMHFA) probe. The experimental measurements were in good agreement with numerical results except in flow with low Reynolds numbers. The wall shear stress increased proximal to the anastomosis in flow from the Penrose tubing (simulating an artery) to the PTFE: graft. In flow from the PTFE graft to the Penrose tubing, low wall shear stress was observed distal to the anastomosis. Abnormal distributions of wall shear stress in the vicinity of the anastomosis, resulting from the compliance mismatch between the graft and the host artery, might be an important factor of ANFH formation and the graft failure. The present study suggests a correlation between regions of the low wall shear stress and the development of anastomotic neointimal fibrous hyperplasia(ANPH) in end-to-end anastomoses. 30523 T00401030523 ^x Air pressure decay(APD) rate and ultrafiltration rate(UFR) tests were performed on new and saline rinsed dialyzers as well as those roused in patients several times. C-DAK 4000 (Cordis Dow) and CF IS-11 (Baxter Travenol) reused dialyzers obtained from the dialysis clinic were used in the present study. The new dialyzers exhibited a relatively flat APD, whereas saline rinsed and reused dialyzers showed considerable amount of decay. C-DAH dialyzers had a larger APD(11.70