Video Analysis System for Action and Emotion Detection by Object with Hierarchical Clustering based Re-ID (계층적 군집화 기반 Re-ID를 활용한 객체별 행동 및 표정 검출용 영상 분석 시스템)
-
- Journal of Intelligence and Information Systems
- /
- v.28 no.1
- /
- pp.89-106
- /
- 2022
Recently, the amount of video data collected from smartphones, CCTVs, black boxes, and high-definition cameras has increased rapidly. According to the increasing video data, the requirements for analysis and utilization are increasing. Due to the lack of skilled manpower to analyze videos in many industries, machine learning and artificial intelligence are actively used to assist manpower. In this situation, the demand for various computer vision technologies such as object detection and tracking, action detection, emotion detection, and Re-ID also increased rapidly. However, the object detection and tracking technology has many difficulties that degrade performance, such as re-appearance after the object's departure from the video recording location, and occlusion. Accordingly, action and emotion detection models based on object detection and tracking models also have difficulties in extracting data for each object. In addition, deep learning architectures consist of various models suffer from performance degradation due to bottlenects and lack of optimization. In this study, we propose an video analysis system consists of YOLOv5 based DeepSORT object tracking model, SlowFast based action recognition model, Torchreid based Re-ID model, and AWS Rekognition which is emotion recognition service. Proposed model uses single-linkage hierarchical clustering based Re-ID and some processing method which maximize hardware throughput. It has higher accuracy than the performance of the re-identification model using simple metrics, near real-time processing performance, and prevents tracking failure due to object departure and re-emergence, occlusion, etc. By continuously linking the action and facial emotion detection results of each object to the same object, it is possible to efficiently analyze videos. The re-identification model extracts a feature vector from the bounding box of object image detected by the object tracking model for each frame, and applies the single-linkage hierarchical clustering from the past frame using the extracted feature vectors to identify the same object that failed to track. Through the above process, it is possible to re-track the same object that has failed to tracking in the case of re-appearance or occlusion after leaving the video location. As a result, action and facial emotion detection results of the newly recognized object due to the tracking fails can be linked to those of the object that appeared in the past. On the other hand, as a way to improve processing performance, we introduce Bounding Box Queue by Object and Feature Queue method that can reduce RAM memory requirements while maximizing GPU memory throughput. Also we introduce the IoF(Intersection over Face) algorithm that allows facial emotion recognized through AWS Rekognition to be linked with object tracking information. The academic significance of this study is that the two-stage re-identification model can have real-time performance even in a high-cost environment that performs action and facial emotion detection according to processing techniques without reducing the accuracy by using simple metrics to achieve real-time performance. The practical implication of this study is that in various industrial fields that require action and facial emotion detection but have many difficulties due to the fails in object tracking can analyze videos effectively through proposed model. Proposed model which has high accuracy of retrace and processing performance can be used in various fields such as intelligent monitoring, observation services and behavioral or psychological analysis services where the integration of tracking information and extracted metadata creates greate industrial and business value. In the future, in order to measure the object tracking performance more precisely, there is a need to conduct an experiment using the MOT Challenge dataset, which is data used by many international conferences. We will investigate the problem that the IoF algorithm cannot solve to develop an additional complementary algorithm. In addition, we plan to conduct additional research to apply this model to various fields' dataset related to intelligent video analysis.
A theroretical study was made on the psychodynamism of somatoform disorder. Somatoform disorder is caused by a defense mechanism of somatization. Somatization is the tendency to react to stimuli(drives, defenses, and conflict between them) physically rather than psychically(Moore, 1990). Ford(1983) said it is a way of life, and Dunbar(1954) said it is the shift of psychic energy toward expression in somatic symptoms. As used by Max Shur(1955), somatization links symptom formation to the regression that may occur in response to acute and chronic conflict. In the neurotic individual psychic conflict often provokes regressive phenomena that may include somatic manifestations characteristic of an earlier developmental phase. Schur calls this resomatization. Pain is the most common example of a somatization reaction to conflict. The pain has an unconscious significance derived from childhood experiences. It is used to win love, to punish misdeeds, as well as a means to amend. Among all pains, chest pain has a special meaning. Generally speaking, 'I have pain in my chest' is about the same as 'I have pain in my mind'. The chest represent the mind, and the mind reminds us about the heart. So we have a high tendency to recognize mental pain as cardiac pain. Kellner(1990) said rage and hostility, especially repressed hostility, are important factors in somatization. In 'Psychoanalytic Observation on Cardiac Pain', psychoanalyst Bacon(1953) presented clinical cases of patients who complained of cardiac pain in a psychoanalytic session that spread from the left side of their chests down their left arms. The pain was from rage and fear which came after their desire to be loved was frustrated by the analyet. She said desires related to cardiac pain were dependency needs and aggressions. Empatic relationship and therapeutic alliances are indispensable to psychotherapy in somatoform disorder. The beginning of therapy is to discover a precipitating event from the time their symptoms have started and to help the patient understand a relation between the symptom and precipitating event. Its remedial process is to find and interpret a intrapsychic conflict shown through the symptoms of the patient. Three cases of somatoform disorder patients treated based on this therapeutic method were introduced. The firt patient, Mr. H, had been suffering from hysterical aphasia with repressed rage as ie psychodynamic cause. An interpretation related to the precipitating event was given by written communication, and he recovered from his aphasia after 3 days of the session. The second patient was a dentist in a cardiac neurosis with agitation and hypochondriasis, whose psychodynamism was caused by a fear that he might lose his father's love. His symptom was also interpreted in relation to the precipitating event. It showed the patient a child-within afraid of losing his father's love. His condition improved after getting a didactic interpretation which told him, to be master of himself, The third patient was a lady transferred from the deparment of internal medicine. She had a frequent and violent fit of chest pains, whose psychodynamic cause was separation anxiety and a rage due to the frustration of dependency needs. Her symptom vanished dramatically when she wore a holler EKG monitor and did not occur during monitoring. By this experience she found her symptom was a psychogenic one, and a therapeutic alliance was formed. later in reguar psychotherapy sessions, she was told the relaton between symptoms and precipitating events. Through this she understood that her separation anxiety was connected to the symptom and she became less terrifide when it occurred. Now she can travel abroad and take well part in social activities.
Background : Insulin-like growth factor (IGF)-binding protein-3 (IGFBP-3) inhibits the proliferation of non-small cell lung cancer (NSCLC) cells by inducing apoptosis. Methods : In this study, we investigated whether hypermethylation of IGFBP-3 promoter play an important role in the loss of IGFBP-3 expression in NSCLC. We also studied the mechanisms that mediate the silencing of IGFBP-3 expression in the cell lines which have hypermethylated IGFBP-3 promoter. Results : The IGFBP-3 promoter has hypermethylation in 7 of 15 (46.7%) NSCLC cell lines and 16 (69.7%) of 23, 7 (77.8%) of 9, 4 (80%) of 5, 4 (66.7 %) of 6, and 6 (100%) of 6 tumor specimens from patients with stage I, II, IIIA, IIIB, and IV NSCLC, respectively. The methylation status correlated with the level of protein and mRNA in NSCLC cell lines. Expression of IGFBP-3 was restored by the demethylating agent 5'-aza-2'-deoxycytidine (5'-aza-dC) in a subset of NSCLC cell lines. The Sp-1/ Sp-3 binding element in the IGFBP-3 promoter, important for promoter activity, was methylated in the NSCLC cell lines which have reduced IGFBP-3 expression and the methylation of this element suppressed the binding of the Sp-1 transcription factor. A ChIP assay showed that the methylation status of the IGFBP-3 promoter influenced the binding of Sp-1, methyl-CpG binding protein-2 (MeCP2), and histone deacetylase (HDAC) to Sp-1/Sp-3 binding element, which were reversed by by 5'-aza-dC. In vitro methylation of the IGFBP-3 promoter containing the Sp-1/Sp-3 binding element significantly reduced promoter activity, which was further suppressed by the overexpression of MeCP2. This reduction in activity was rescued by 5'-aza-dC. Conclusion : These findings indicate that hypermethylation of the IGFBP-3 promoter is one mechanism by which IGFBP-3 expression is silenced and MeCP2, with recruitment of HDAC, may play a role in silencing of IGFBP-3 expression. The frequency of this abnormality is also associated with advanced stages among the patients with NSCLC, suggesting that IGFBP-3 plays an important role in lung carcinogenesis/progression and that the promoter methylation status of IGFBP-3 may be a marker for early molecular detection and/or for monitoring chemoprevention efforts.
The International Atomic Energy Agency's Statute in Article III.A.5 allows it“to establish and administer safeguards designed to ensure that special fissionable and other materials, services, equipment, facilities and information made available by the Agency or at its request or under its supervision or control are not used in such a way as to further any military purpose; and to apply safeguards, at the request of the parties, to any bilateral or multilateral arrangement, or at the request of a State, to any of that State's activities in the field of atomic energy”. Safeguards are essentially a technical means of verifying the fulfilment of political obligations undertaken by States and given a legal force in international agreements relating to the peaceful uses of nuclear energy. The main political objectives are: to assure the international community that States are complying with their non-proliferation and other peaceful undertakings; and to deter (a) the diversion of afeguarded nuclear materials to the production of nuclear explosives or for military purposes and (b) the misuse of safeguarded facilities with the aim of producing unsafeguarded nuclear material. It is clear that no international safeguards system can physically prevent diversion. The IAEA safeguards system is basically a verification measure designed to provide assurance in those cases in which diversion has not occurred. Verification is accomplished by two basic means: material accountancy and containment and surveillance measures. Nuclear material accountancy is the fundamental IAEA safeguards mechanism, while containment and surveillance serve as important complementary measures. Material accountancy refers to a collection of measurements and other determinations which enable the State and the Agency to maintain a current picture of the location and movement of nuclear material into and out of material balance areas, i. e. areas where all material entering or leaving is measurab e. A containment measure is one that is designed by taking advantage of structural characteristics, such as containers, tanks or pipes, etc. To establish the physical integrity of an area or item by preventing the undetected movement of nuclear material or equipment. Such measures involve the application of tamper-indicating or surveillance devices. Surveillance refers to both human and instrumental observation aimed at indicating the movement of nuclear material. The verification process consists of three over-lapping elements: (a) Provision by the State of information such as - design information describing nuclear installations; - accounting reports listing nuclear material inventories, receipts and shipments; - documents amplifying and clarifying reports, as applicable; - notification of international transfers of nuclear material. (b) Collection by the IAEA of information through inspection activities such as - verification of design information - examination of records and repo ts - measurement of nuclear material - examination of containment and surveillance measures - follow-up activities in case of unusual findings. (c) Evaluation of the information provided by the State and of that collected by inspectors to determine the completeness, accuracy and validity of the information provided by the State and to resolve any anomalies and discrepancies. To design an effective verification system, one must identify possible ways and means by which nuclear material could be diverted from peaceful uses, including means to conceal such diversions. These theoretical ways and means, which have become known as diversion strategies, are used as one of the basic inputs for the development of safeguards procedures, equipment and instrumentation. For analysis of implementation strategy purposes, it is assumed that non-compliance cannot be excluded a priori and that consequently there is a low but non-zero probability that a diversion could be attempted in all safeguards ituations. An important element of diversion strategies is the identification of various possible diversion paths; the amount, type and location of nuclear material involved, the physical route and conversion of the material that may take place, rate of removal and concealment methods, as appropriate. With regard to the physical route and conversion of nuclear material the following main categories may be considered: - unreported removal of nuclear material from an installation or during transit - unreported introduction of nuclear material into an installation - unreported transfer of nuclear material from one material balance area to another - unreported production of nuclear material, e. g. enrichment of uranium or production of plutonium - undeclared uses of the material within the installation. With respect to the amount of nuclear material that might be diverted in a given time (the diversion rate), the continuum between the following two limiting cases is cons dered: - one significant quantity or more in a short time, often known as abrupt diversion; and - one significant quantity or more per year, for example, by accumulation of smaller amounts each time to add up to a significant quantity over a period of one year, often called protracted diversion. Concealment methods may include: - restriction of access of inspectors - falsification of records, reports and other material balance areas - replacement of nuclear material, e. g. use of dummy objects - falsification of measurements or of their evaluation - interference with IAEA installed equipment.As a result of diversion and its concealment or other actions, anomalies will occur. All reasonable diversion routes, scenarios/strategies and concealment methods have to be taken into account in designing safeguards implementation strategies so as to provide sufficient opportunities for the IAEA to observe such anomalies. The safeguards approach for each facility will make a different use of these procedures, equipment and instrumentation according to the various diversion strategies which could be applicable to that facility and according to the detection and inspection goals which are applied. Postulated pathways sets of scenarios comprise those elements of diversion strategies which might be carried out at a facility or across a State's fuel cycle with declared or undeclared activities. All such factors, however, contain a degree of fuzziness that need a human judgment to make the ultimate conclusion that all material is being used for peaceful purposes. Safeguards has been traditionally based on verification of declared material and facilities using material accountancy as a fundamental measure. The strength of material accountancy is based on the fact that it allows to detect any diversion independent of the diversion route taken. Material accountancy detects a diversion after it actually happened and thus is powerless to physically prevent it and can only deter by the risk of early detection any contemplation by State authorities to carry out a diversion. Recently the IAEA has been faced with new challenges. To deal with these, various measures are being reconsidered to strengthen the safeguards system such as enhanced assessment of the completeness of the State's initial declaration of nuclear material and installations under its jurisdiction enhanced monitoring and analysis of open information and analysis of open information that may indicate inconsistencies with the State's safeguards obligations. Precise information vital for such enhanced assessments and analyses is normally not available or, if available, difficult and expensive collection of information would be necessary. Above all, realistic appraisal of truth needs sound human judgment.