• 인터넷정보학회논문지
• /
• 제22권4호
• /
• pp.13-28
• /
• 2021

강화학습은 한 환경에서 에이전트가 정책에 따라 액션을 취하고 보상 함수를 통해 액션을 평가 및 정책 최적화 과정을 반복하는 Closed-Loop 구조로 이루어진 알고리즘이다. 이러한 강화학습의 주요 장점은 액션의 품질을 평가하고 정책을 지속적으로 최적화 하는 것이다. 따라서, 강화학습은 지능형 시스템, 자율제어 시스템 개발에 효과적으로 활용될 수 있다. 기존의 강화학습은, 단일 정책, 단일 보상함수 및 비교적 단순한 정책 업데이트 기법을 제한적인 문제에 대해 제시하고 적용하였다. 본 논문에서는 구성요소의 복수성을 지원하는 확장된 강화학습 모델을 제안한다. 제안되는 확정 강화학습의 주요 구성 요소들을 정의하고, 그들의 컴퓨팅 모델을 포함하는 정형 모델을 제시한다. 또한, 이 정형모델을 기반으로 시스템 개발을 위한 설계 기법을 제시한다. 제안한 모델을 기반으로 자율 최적화 자동차 내비게이터 시스템에 적용 및 실험을 진행한다. 제시된 정형 모델과 설계 기법을 적용한 사례연구로, 복수의 자동차들이 최적 목적지에 단 시간에 도착할 수 있는 진화된 내비게이터 시스템 설계 및 구현을 진행한다.

• 한국정보기술학회 영문논문지
• /
• 제9권1호
• /
• pp.127-134
• /
• 2019

In terms of deep Reinforcement Learning (RL), exploration can be worked stochastically in the action of a state space. On the other hands, exploitation can be done the proportion of well generalization behaviors. The balance of exploration and exploitation is extremely important for better results. The randomly selected action with ε-greedy for exploration has been regarded as a de facto method. There is an alternative method to add noise parameters into a neural network for richer exploration. However, it is not easy to predict or detect over-fitting with the stochastically exploration in the perturbed neural network. Moreover, the well-trained agents in RL do not necessarily prevent or detect over-fitting in the neural network. Therefore, we suggest a novel design of a deep RL by the balance of the exploration with drop-out to reduce over-fitting in the perturbed neural networks.

• 한국항행학회논문지
• /
• 제26권4호
• /
• pp.226-234
• /
• 2022

자율주행체와 네트워크기반 제어 기술의 발달에 따라서, 하나의 에이전트를 제어하는 것을 넘어서 다수의 이동체를 분산 제어하는데 사용 가능한 다중 에이전트의 컨센서스 제어에 대한 관심과 연구가 증가하고 있다. 컨센서스 제어는 분산형 제어이기 때문에, 정보 교환은 실제 시스템에서 지연을 가지게 된다. 또한, 시스템에 대한 모델을 정확히 수식적으로 표현하는데 있어서 한계를 갖는다. 이런 한계를 극복하는 방법 중에 하나로서 강화 학습 기반 컨센서스 알고리즘이 개발되었지만, 불확실성이 큰 환경에서 느린 수렴을 갖는 경우가 자주 발생하는 특징을 보이고 있다. 따라서, 이 논문에서는 불확실성에 강인한 특성을 갖는 슬라이딩 모드제어를 강화학습과 결합한 슬라이딩 강화학습 알고리즘을 제안한다. 제안 알고리즘은 기존의 강화학습 기반 컨센서스 알고리즘의 제어 신호에 슬라이딩 모드 제어 구조를 추가하고, 시스템의 상태 정보를 슬라이딩 변수를 추가하여 확장한다. 모의실험 결과 다양한 시변 지연과 왜란에 대한 정보가 주어지지 않았을 때에 슬라이딩 강화학습 알고리즘은 모델기반의 알고리즘과 유사한 성능을 보이면서, 기존의 강화학습에 비해서 안정적이면서 우수한 성능을 보여준다.

• 한국공간구조학회논문집
• /
• 제23권2호
• /
• pp.29-36
• /
• 2023

Reinforcement learning (RL) is widely applied to various engineering fields. Especially, RL has shown successful performance for control problems, such as vehicles, robotics, and active structural control system. However, little research on application of RL to optimal structural design has conducted to date. In this study, the possibility of application of RL to structural design of reinforced concrete (RC) beam was investigated. The example of RC beam structural design problem introduced in previous study was used for comparative study. Deep q-network (DQN) is a famous RL algorithm presenting good performance in the discrete action space and thus it was used in this study. The action of DQN agent is required to represent design variables of RC beam. However, the number of design variables of RC beam is too many to represent by the action of conventional DQN. To solve this problem, multi-agent DQN was used in this study. For more effective reinforcement learning process, DDQN (Double Q-Learning) that is an advanced version of a conventional DQN was employed. The multi-agent of DDQN was trained for optimal structural design of RC beam to satisfy American Concrete Institute (318) without any hand-labeled dataset. Five agents of DDQN provides actions for beam with, beam depth, main rebar size, number of main rebar, and shear stirrup size, respectively. Five agents of DDQN were trained for 10,000 episodes and the performance of the multi-agent of DDQN was evaluated with 100 test design cases. This study shows that the multi-agent DDQN algorithm can provide successfully structural design results of RC beam.

• 한국자원식물학회:학술대회논문집
• /
• 한국자원식물학회 2023년도 임시총회 및 춘계학술대회
• /
• pp.38-38
• /
• 2023

Sorghum (Sorghum bicolor L.) is a promising biomass crop with a high lignocellulose content. This study aimed to select high salt-tolerance sorghum lines for cultivation on reclaimed land. Using 7-day seedlings of the sorghum population consisted of 71 radiation-derived mutants (M₂ to M₆) and 33 genetic resources, survival rate (SR), plant height (PH), root length (RL), fresh weight (FW), and chlorophyll content (CC) were measured for two weeks after 102 mM (0.6%) NaCl treatment. Furthermore, the characteristics of the sorghum population were confirmed using correlation analysis, PCA (principal component analysis), and the FCE (fuzzy comprehensive evaluation) method. Under 102 mM NaCl conditions, SR ranged from 4.9 (IS645-200-6) to 82.4% (KLSo79125-200-1), with an average of 49.9%. PH varied from 7.5 (Mesusu-100-2) to 33.2 cm (DINE-A-MITE-100-2-10), with an average of 20.4 cm. RL ranged from 1.0 (IS645-200-1) to 17.0 cm (30-100-2), with an average of 7.7 cm. FW varied from 0.1 (IS645-200-6) to 4.5 g/plant (DINE-A-MITE-100-2-10), with an average of 2.1 g/plant. CC ranged from 0.9 (DINE-A-MITE-100-2-2) to 3.1 mg/g (IS12937), with an average of 1.7 mg/g. An overall positive correlation, with SR and FW (r = 0.86, P < 0.01), and FW and CC (r = 0.79, P < 0.01), was shown by correlation analysis. Among the five traits, two principal components were extracted by PCA analysis. PC1 was significantly associated with FW, while PC2 was highly involved with RL. To evaluate the salt tolerance level of the sorghum population when an FCE based on trait data was performed, MFV (membership function value) was 0.68. As a result of compiling the MFV of each line, eight lines with MFV > 0.68 were selected. Ultimately, the radiation-derived mutant lines, DINE-A-MITE-100-2-10 and DINE-A-MITE-100-2-12 were selected as salt-tolerant sorghum lines. The results are expected to inform salt-tolerant sorghum breeding programs, and the high salt-tolerance sorghum lines might be advantageous for cultivation on reclaimed land.

• Journal of Plant Biotechnology
• /
• 제4권2호
• /
• pp.59-65
• /
• 2002

ADP-glucose pyrophosphorylase (AGPase) catalyzes the key regulatory step in starch biosynthesis. Two cDNA clones encoding AGPase subunits were isolated from the leaf cDNA library of Chinese cabbage (Brassica campestris L. spp. pekinensis). One was designated as BCAGPS for the small subunit and the other as BCAGPL for the large subunit. Both cDNAs have uninterrupted open reading frames deriving 57 kDa and 63 kDa polypeptides for BCAGPS and BCAGPL, respectively, which showed significant similarity to those of other dicot plants. Also, However, the deduced amino acid sequence of BCAGPL has a unique feature. That is, it contains two regions (Rl and R2) lacking in all other plant enzymes. This is the first report of BCAGPL containing Rl and R2 among plant large subunits as well as small subunits. From the genomic Southern analysis and BAC library screening, we inferred the genomic status of BCAGPS and BCAGPL gene.

• Radiation Oncology Journal
• /
• 제35권1호
• /
• pp.65-70
• /
• 2017

Purpose: To evaluate the utility of implanted surgical clips for detecting interfractional errors in the treatment of hepatobiliary and pancreatic cancer with postoperative radiotherapy (PORT). Methods and Materials: Twenty patients had been treated with PORT for locally advanced hepatobiliary or pancreatic cancer, from November 2014 to April 2016. Patients underwent computed tomography simulation and were treated in expiratory breathing phase. During treatment, orthogonal kilovoltage (kV) imaging was taken twice a week, and isocenter shifts were made to match bony anatomy. The difference in position of clips between kV images and digitally reconstructed radiographs was determined. Clips were consist of 3 proximal clips (clip_p, ${\leq}2cm$) and 3 distal clips (clip_d, >2 cm), which were classified according to distance from treatment center. The interfractional displacements of clips were measured in the superior-inferior (SI), anterior-posterior (AP), and right-left (RL) directions. Results: The translocation of clip was well correlated with diaphragm movement in 90.4% (190/210) of all images. The clip position errors greater than 5 mm were observed in 26.0% in SI, 1.8% in AP, and 5.4% in RL directions, respectively. Moreover, the clip position errors greater than 10 mm were observed in 1.9% in SI, 0.2% in AP, and 0.2% in RL directions, despite respiratory control. Conclusion: Quantitative analysis of surgical clip displacement reflect respiratory motion, setup errors and postoperative change of intraabdominal organ position. Furthermore, position of clips is distinguished easily in verification images. The identification of the surgical clip position may lead to a significant improvement in the accuracy of upper abdominal radiation therapy.

• The Journal of Advanced Prosthodontics
• /
• 제13권4호
• /
• pp.246-257
• /
• 2021

PURPOSE. This study aimed to investigate the Vickers Hardness Number (VHN) of light- and dual cured resin cements cured through monolithic zirconia specimens (VITA YZ) of various translucencies: translucent (T); high translucent (HT); super translucent (ST); and extra translucent (XT) at 0, 24, and 48 h after curing. MATERIALS AND METHODS. Four zirconia specimens from each translucency were prepared. Two light-cured resin cements (Variolink N LC; VL and RelyX Veneer; RL) and two dual-cured resin cements (Variolink N DC; VD and RelyX U200; RD) were used. The cement was mixed and loaded in a mold and cured for 20 s through the zirconia specimen. The upper surface of cements was tested for VHN using a microhardness tester at 0, 24, and 48 h after curing. The VHN were analyzed using two-way repeated, Brown-Forsythe ANOVA with Games Howell post-hoc analysis and independent t-tests (P < .05). RESULTS. All cements showed significantly higher VHN from 0 h to 24 h (P < .001). At 48 h, the VHN of light-cured cements were significantly lower when cured under the T groups than under XT groups (P = .001 in VL, P = .014 in RL). At each post curing time of each translucency, VD showed higher VHN than VL (P < .05), and RD also showed higher VHN than RL (P < .05). CONCLUSION. The translucency of zirconia has an effect on the VHN for light-cured resin cements, but has no effect on dual-cured resin cements. Dual-cured resin cement exhibited higher VHN than the light-cured resin cement from the same manufacturer. All resin cements showed significantly higher VHN from 0 h to 24 h.

• 한국측량학회지
• /
• 제29권6호
• /
• pp.593-599
• /
• 2011

지리정보에 대한 사용자의 활용도가 높아지면서 더 새롭고 고품질인 지리정보 자료에 대한 요구 역시 함께 증가하고 있다. 새로운 콘텐츠를 제작하는데 있어 기 구축된 자료간의 융합을 통한 방법은 기존의 자료들이 가진 장점만을 취하여 새로운 지리정보 콘텐츠를 생성할 수 있다는 점에서 효율적이다. 그러나, 서로 다른 자료를 융합하면 위치편차와 형상불일치가 나타나는데 이는 이종의 자료가 가진 정보의 정확도는 그대로 유지한 채 자료 간 불일치되는 부분을 개선하는 합성 기술로 해결이 가능하다. 본 연구에서는 선행연구를 통해 위치편차가 최소화된 도로지도와 항공사진간에 형상불일치를 제거하는 것을 목적으로 하여 항공사진에서 도로 후보 영상을 생성한 후 이를 도로일 가능성으로 표현한 그래프를 제작하였다. 여기서 도로일 가능성이 높은 것만을 추출하여 합성 시 형상불일치를 제거할 수 있었다.

• Nuclear Engineering and Technology
• /
• 제56권4호
• /
• pp.1472-1479
• /
• 2024

Cyber-Physical Energy Systems (CPESs) integrate cyber and hardware components to ensure a reliable and safe physical power production and supply. Renewable Energy Sources (RESs) add uncertainty to energy demand that can be dealt with flexible operation (e.g., load-following) of CPES; at the same time, scenarios that could result in severe consequences due to both component stochastic failures and aging of the cyber system of CPES (commonly overlooked) must be accounted for Operation & Maintenance (O&M) planning. In this paper, we make use of Deep Reinforcement Learning (DRL) to search for the optimal O&M strategy that, not only considers the actual system hardware components health conditions and their Remaining Useful Life (RUL), but also the possible accident scenarios caused by the failures and the aging of the hardware and the cyber components, respectively. The novelty of the work lies in embedding the cyber aging model into the CPES model of production planning and failure process; this model is used to help the RL agent, trained with Proximal Policy Optimization (PPO) and Imitation Learning (IL), finding the proper rejuvenation timing for the cyber system accounting for the uncertainty of the cyber system aging process. An application is provided, with regards to the Advanced Lead-cooled Fast Reactor European Demonstrator (ALFRED).