• 제목/요약/키워드: Deep Recurrent Q Network

검색결과 7건 처리시간 0.021초

Application of Deep Recurrent Q Network with Dueling Architecture for Optimal Sepsis Treatment Policy

  • Do, Thanh-Cong;Yang, Hyung Jeong;Ho, Ngoc-Huynh
    • 스마트미디어저널
    • /
    • 제10권2호
    • /
    • pp.48-54
    • /
    • 2021
  • Sepsis is one of the leading causes of mortality globally, and it costs billions of dollars annually. However, treating septic patients is currently highly challenging, and more research is needed into a general treatment method for sepsis. Therefore, in this work, we propose a reinforcement learning method for learning the optimal treatment strategies for septic patients. We model the patient physiological time series data as the input for a deep recurrent Q-network that learns reliable treatment policies. We evaluate our model using an off-policy evaluation method, and the experimental results indicate that it outperforms the physicians' policy, reducing patient mortality up to 3.04%. Thus, our model can be used as a tool to reduce patient mortality by supporting clinicians in making dynamic decisions.

순환인공신경망을 활용한 터널굴착면 전방 Q값 예측에 관한 연구 (Study on Q-value prediction ahead of tunnel excavation face using recurrent neural network)

  • 홍창호;김진;류희환;조계춘
    • 한국터널지하공간학회 논문집
    • /
    • 제22권3호
    • /
    • pp.239-248
    • /
    • 2020
  • 터널 굴착 시 정확한 암반 분류는 적합한 지보패턴을 설치하는 데 도움을 준다. 암반의 분류를 위해 주로 RMR (Rock Mass Ration)과 Q값을 산정하여 수행되며, 페이스 매핑(face mapping)을 바탕으로 산정된다. 점보드릴 및 프로브드릴의 기계 데이터을 활용하거나 딥러닝을 활용한 굴착면 사진 분석 등의 방법이 암반등급 분류를 예측하기 위해 사용되고 있으나, 분석 시 오랜 시간이 소요되거나, 굴착면 전방의 암반등급을 파악할 수 없다는 점에서 한계를 갖는다. 본 연구에서는 순환인공신경망(Recurrent neural network, RNN)을 활용하여 굴착면 전방의 Q값을 예측하는 방법을 개발하였고 페이스 매핑으로부터 획득한 Q값과 비교/검증하였다. 4,600여개의 굴착면 데이터 중 70%를 학습에 활용하였고, 나머지 30%는 검증에 사용하였다. 학습의 횟수와 학습에 활용한 이전굴착면의 개수를 변경하여 학습을 수행하였다. 예측된 Q값과 실제 Q값의 유사도는 RMSE (root mean square error)를 기준으로 비교하였다. 현재 굴착면과 바로 직전의 굴착면의 Q값을 활용하여 600회 학습하여 예측한 Q값의 RMSE값이 가장 작은 것을 확인하였다. 본 연구의 결과는 학습에 사용한 데이터 값 등이 변화하는 경우 변화할 수 있으나 터널에서의 이전 지반상태가 앞으로의 지반상태에 영향을 미치는 시스템을 이해하고, 이를 통해 터널 굴착면 전방의 Q값의 예측이 가능할 것으로 판단된다.

심층 순환 Q 네트워크 기반 목적 지향 대화 시스템 (Goal Oriented Dialogue System Based on Deep Recurrent Q Network)

  • 박건우;김학수
    • 한국정보과학회 언어공학연구회:학술대회논문집(한글 및 한국어 정보처리)
    • /
    • 한국정보과학회언어공학연구회 2018년도 제30회 한글 및 한국어 정보처리 학술대회
    • /
    • pp.147-150
    • /
    • 2018
  • 목적 지향 대화 시스템은 자연어 이해, 대화 관리자, 자연어 생성과 같은 세분화 모델들의 결합으로 이루어져있어 하위 모델에 대한 오류 전파에 취약하다. 이러한 문제점을 해결하기 위해 자연어 이해 모델과 대화 관리자를 하나의 네트워크로 구성하고 오류에 강건한 심층 Q 네트워크를 제안한다. 본 논문에서는 대화의 전체 흐름을 파악 할 수 있는 순환 신경망인 LSTM에 심층 Q 네트워크 적용한 심층 순환 Q 네트워크 기반 목적 지향 대화 시스템을 제안한다. 실험 결과, 제안한 심층 순환 Q 네트워크는 LSTM, 심층 Q 네트워크보다 각각 정밀도 1.0%p, 6.7%p 높은 성능을 보였다.

  • PDF

딥 러닝을 이용한 자동 댓글 생성에 관한 연구 (A Study on Automatic Comment Generation Using Deep Learning)

  • 최재용;성소윤;김경철
    • 한국게임학회 논문지
    • /
    • 제18권5호
    • /
    • pp.83-92
    • /
    • 2018
  • 최근 다수의 분야에서 딥 러닝을 통한 연구 성과들이 사람의 판단력에 근접하는 결과를 보여주고 있다. 그리고 게임 산업에서는 온라인 커뮤니티, SNS의 활성화가 게임 흥행 여부를 결정할 정도로 중요성이 높아지고 있다. 본 연구는 딥 러닝을 이용해 온라인 커뮤니티, SNS에서 활동할 수 있는 시스템을 구성하고, 온라인 공간에서 사람들이 작성한 텍스트를 읽고 그에 대한 반응을 생성하고 스케쥴에 따라 트위터에 올리는 것을 목표로 한다. 순환 신경망(Recurrent Neural Network)을 이용해 텍스트를 생성하고 글 작성 스케쥴을 생성하는 모델들을 구성했고, 생성한 시각에 맞춰 모델들에 뉴스 제목을 입력해 댓글을 출력 받고 트위터에 작성하는 프로그램을 구현했다. 본 연구결과는 온라인 게임 커뮤니티 활성화, Q&A 서비스 등에 적용이 가능할 것으로 예상된다.

Application of Deep Learning: A Review for Firefighting

  • Shaikh, Muhammad Khalid
    • International Journal of Computer Science & Network Security
    • /
    • 제22권5호
    • /
    • pp.73-78
    • /
    • 2022
  • The aim of this paper is to investigate the prevalence of Deep Learning in the literature on Fire & Rescue Service. It is found that deep learning techniques are only beginning to benefit the firefighters. The popular areas where deep learning techniques are making an impact are situational awareness, decision making, mental stress, injuries, well-being of the firefighter such as his sudden fall, inability to move and breathlessness, path planning by the firefighters while getting to an fire scene, wayfinding, tracking firefighters, firefighter physical fitness, employment, prediction of firefighter intervention, firefighter operations such as object recognition in smoky areas, firefighter efficacy, smart firefighting using edge computing, firefighting in teams, and firefighter clothing and safety. The techniques that were found applied in firefighting were Deep learning, Traditional K-Means clustering with engineered time and frequency domain features, Convolutional autoencoders, Long Short-Term Memory (LSTM), Deep Neural Networks, Simulation, VR, ANN, Deep Q Learning, Deep learning based on conditional generative adversarial networks, Decision Trees, Kalman Filters, Computational models, Partial Least Squares, Logistic Regression, Random Forest, Edge computing, C5 Decision Tree, Restricted Boltzmann Machine, Reinforcement Learning, and Recurrent LSTM. The literature review is centered on Firefighters/firemen not involved in wildland fires. The focus was also not on the fire itself. It must also be noted that several deep learning techniques such as CNN were mostly used in fire behavior, fire imaging and identification as well. Those papers that deal with fire behavior were also not part of this literature review.

지도학습과 강화학습을 이용한 준능동 중간층면진시스템의 최적설계 (Optimal Design of Semi-Active Mid-Story Isolation System using Supervised Learning and Reinforcement Learning)

  • 강주원;김현수
    • 한국공간구조학회논문집
    • /
    • 제21권4호
    • /
    • pp.73-80
    • /
    • 2021
  • A mid-story isolation system was proposed for seismic response reduction of high-rise buildings and presented good control performance. Control performance of a mid-story isolation system was enhanced by introducing semi-active control devices into isolation systems. Seismic response reduction capacity of a semi-active mid-story isolation system mainly depends on effect of control algorithm. AI(Artificial Intelligence)-based control algorithm was developed for control of a semi-active mid-story isolation system in this study. For this research, an practical structure of Shiodome Sumitomo building in Japan which has a mid-story isolation system was used as an example structure. An MR (magnetorheological) damper was used to make a semi-active mid-story isolation system in example model. In numerical simulation, seismic response prediction model was generated by one of supervised learning model, i.e. an RNN (Recurrent Neural Network). Deep Q-network (DQN) out of reinforcement learning algorithms was employed to develop control algorithm The numerical simulation results presented that the DQN algorithm can effectively control a semi-active mid-story isolation system resulting in successful reduction of seismic responses.

CAB: Classifying Arrhythmias based on Imbalanced Sensor Data

  • Wang, Yilin;Sun, Le;Subramani, Sudha
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • 제15권7호
    • /
    • pp.2304-2320
    • /
    • 2021
  • Intelligently detecting anomalies in health sensor data streams (e.g., Electrocardiogram, ECG) can improve the development of E-health industry. The physiological signals of patients are collected through sensors. Timely diagnosis and treatment save medical resources, promote physical health, and reduce complications. However, it is difficult to automatically classify the ECG data, as the features of ECGs are difficult to extract. And the volume of labeled ECG data is limited, which affects the classification performance. In this paper, we propose a Generative Adversarial Network (GAN)-based deep learning framework (called CAB) for heart arrhythmia classification. CAB focuses on improving the detection accuracy based on a small number of labeled samples. It is trained based on the class-imbalance ECG data. Augmenting ECG data by a GAN model eliminates the impact of data scarcity. After data augmentation, CAB classifies the ECG data by using a Bidirectional Long Short Term Memory Recurrent Neural Network (Bi-LSTM). Experiment results show a better performance of CAB compared with state-of-the-art methods. The overall classification accuracy of CAB is 99.71%. The F1-scores of classifying Normal beats (N), Supraventricular ectopic beats (S), Ventricular ectopic beats (V), Fusion beats (F) and Unclassifiable beats (Q) heartbeats are 99.86%, 97.66%, 99.05%, 98.57% and 99.88%, respectively. Unclassifiable beats (Q) heartbeats are 99.86%, 97.66%, 99.05%, 98.57% and 99.88%, respectively.