• Journal of Internet Computing and Services
• /
• v.22 no.4
• /
• pp.13-28
• /
• 2021

Reinforcement Learning (RL) is a machine learning algorithm that repeat the closed-loop process that agents perform actions specified by the policy, the action is evaluated with a reward function, and the policy gets updated accordingly. The key benefit of RL is the ability to optimze the policy with action evaluation. Hence, it can effectively be applied to developing advanced intelligent systems and autonomous systems. Conventional RL incoporates a single policy, a reward function, and relatively simple policy update, and hence its utilization was limited. In this paper, we propose an extended RL model that considers multiple instances of RL elements. We define a formal model of the key elements and their computing model of the extended RL. Then, we propose design methods for applying to system development. As a case stud of applying the proposed formal model and the design methods, we present the design and implementation of an advanced car navigator system that guides multiple cars to reaching their destinations efficiently.

• Journal of Advanced Information Technology and Convergence
• /
• v.9 no.1
• /
• pp.127-134
• /
• 2019

In terms of deep Reinforcement Learning (RL), exploration can be worked stochastically in the action of a state space. On the other hands, exploitation can be done the proportion of well generalization behaviors. The balance of exploration and exploitation is extremely important for better results. The randomly selected action with ε-greedy for exploration has been regarded as a de facto method. There is an alternative method to add noise parameters into a neural network for richer exploration. However, it is not easy to predict or detect over-fitting with the stochastically exploration in the perturbed neural network. Moreover, the well-trained agents in RL do not necessarily prevent or detect over-fitting in the neural network. Therefore, we suggest a novel design of a deep RL by the balance of the exploration with drop-out to reduce over-fitting in the perturbed neural networks.

• Journal of Advanced Navigation Technology
• /
• v.26 no.4
• /
• pp.226-234
• /
• 2022

With advances in autonomous vehicles and networked control, there is a growing interest in the consensus control of a multi-agents system to control multi-agents with distributed control beyond the control of a single agent. Since consensus control is a distributed control, it is bound to have delay in a practical system. In addition, it is often difficult to have a very accurate mathematical model for a system. Even though a reinforcement learning (RL) method was developed to deal with these issues, it often experiences slow convergence in the presence of large uncertainties. Thus, we propose a slide RL which combines the sliding mode control with RL to be robust to the uncertainties. The structure of a sliding mode control is introduced to the action in RL while an auxiliary sliding variable is included in the state information. Numerical simulation results show that the slide RL provides comparable performance to the model-based consensus control in the presence of unknown time-varying delay and disturbance while outperforming existing state-of-the-art RL-based consensus algorithms.

• Journal of Korean Association for Spatial Structures
• /
• v.23 no.2
• /
• pp.29-36
• /
• 2023

Reinforcement learning (RL) is widely applied to various engineering fields. Especially, RL has shown successful performance for control problems, such as vehicles, robotics, and active structural control system. However, little research on application of RL to optimal structural design has conducted to date. In this study, the possibility of application of RL to structural design of reinforced concrete (RC) beam was investigated. The example of RC beam structural design problem introduced in previous study was used for comparative study. Deep q-network (DQN) is a famous RL algorithm presenting good performance in the discrete action space and thus it was used in this study. The action of DQN agent is required to represent design variables of RC beam. However, the number of design variables of RC beam is too many to represent by the action of conventional DQN. To solve this problem, multi-agent DQN was used in this study. For more effective reinforcement learning process, DDQN (Double Q-Learning) that is an advanced version of a conventional DQN was employed. The multi-agent of DDQN was trained for optimal structural design of RC beam to satisfy American Concrete Institute (318) without any hand-labeled dataset. Five agents of DDQN provides actions for beam with, beam depth, main rebar size, number of main rebar, and shear stirrup size, respectively. Five agents of DDQN were trained for 10,000 episodes and the performance of the multi-agent of DDQN was evaluated with 100 test design cases. This study shows that the multi-agent DDQN algorithm can provide successfully structural design results of RC beam.

• Proceedings of the Plant Resources Society of Korea Conference
• /
• 2023.04a
• /
• pp.38-38
• /
• 2023

Sorghum (Sorghum bicolor L.) is a promising biomass crop with a high lignocellulose content. This study aimed to select high salt-tolerance sorghum lines for cultivation on reclaimed land. Using 7-day seedlings of the sorghum population consisted of 71 radiation-derived mutants (M₂ to M₆) and 33 genetic resources, survival rate (SR), plant height (PH), root length (RL), fresh weight (FW), and chlorophyll content (CC) were measured for two weeks after 102 mM (0.6%) NaCl treatment. Furthermore, the characteristics of the sorghum population were confirmed using correlation analysis, PCA (principal component analysis), and the FCE (fuzzy comprehensive evaluation) method. Under 102 mM NaCl conditions, SR ranged from 4.9 (IS645-200-6) to 82.4% (KLSo79125-200-1), with an average of 49.9%. PH varied from 7.5 (Mesusu-100-2) to 33.2 cm (DINE-A-MITE-100-2-10), with an average of 20.4 cm. RL ranged from 1.0 (IS645-200-1) to 17.0 cm (30-100-2), with an average of 7.7 cm. FW varied from 0.1 (IS645-200-6) to 4.5 g/plant (DINE-A-MITE-100-2-10), with an average of 2.1 g/plant. CC ranged from 0.9 (DINE-A-MITE-100-2-2) to 3.1 mg/g (IS12937), with an average of 1.7 mg/g. An overall positive correlation, with SR and FW (r = 0.86, P < 0.01), and FW and CC (r = 0.79, P < 0.01), was shown by correlation analysis. Among the five traits, two principal components were extracted by PCA analysis. PC1 was significantly associated with FW, while PC2 was highly involved with RL. To evaluate the salt tolerance level of the sorghum population when an FCE based on trait data was performed, MFV (membership function value) was 0.68. As a result of compiling the MFV of each line, eight lines with MFV > 0.68 were selected. Ultimately, the radiation-derived mutant lines, DINE-A-MITE-100-2-10 and DINE-A-MITE-100-2-12 were selected as salt-tolerant sorghum lines. The results are expected to inform salt-tolerant sorghum breeding programs, and the high salt-tolerance sorghum lines might be advantageous for cultivation on reclaimed land.

• Journal of Plant Biotechnology
• /
• v.4 no.2
• /
• pp.59-65
• /
• 2002

ADP-glucose pyrophosphorylase (AGPase) catalyzes the key regulatory step in starch biosynthesis. Two cDNA clones encoding AGPase subunits were isolated from the leaf cDNA library of Chinese cabbage (Brassica campestris L. spp. pekinensis). One was designated as BCAGPS for the small subunit and the other as BCAGPL for the large subunit. Both cDNAs have uninterrupted open reading frames deriving 57 kDa and 63 kDa polypeptides for BCAGPS and BCAGPL, respectively, which showed significant similarity to those of other dicot plants. Also, However, the deduced amino acid sequence of BCAGPL has a unique feature. That is, it contains two regions (Rl and R2) lacking in all other plant enzymes. This is the first report of BCAGPL containing Rl and R2 among plant large subunits as well as small subunits. From the genomic Southern analysis and BAC library screening, we inferred the genomic status of BCAGPS and BCAGPL gene.

• Radiation Oncology Journal
• /
• v.35 no.1
• /
• pp.65-70
• /
• 2017

Purpose: To evaluate the utility of implanted surgical clips for detecting interfractional errors in the treatment of hepatobiliary and pancreatic cancer with postoperative radiotherapy (PORT). Methods and Materials: Twenty patients had been treated with PORT for locally advanced hepatobiliary or pancreatic cancer, from November 2014 to April 2016. Patients underwent computed tomography simulation and were treated in expiratory breathing phase. During treatment, orthogonal kilovoltage (kV) imaging was taken twice a week, and isocenter shifts were made to match bony anatomy. The difference in position of clips between kV images and digitally reconstructed radiographs was determined. Clips were consist of 3 proximal clips (clip_p, ${\leq}2cm$) and 3 distal clips (clip_d, >2 cm), which were classified according to distance from treatment center. The interfractional displacements of clips were measured in the superior-inferior (SI), anterior-posterior (AP), and right-left (RL) directions. Results: The translocation of clip was well correlated with diaphragm movement in 90.4% (190/210) of all images. The clip position errors greater than 5 mm were observed in 26.0% in SI, 1.8% in AP, and 5.4% in RL directions, respectively. Moreover, the clip position errors greater than 10 mm were observed in 1.9% in SI, 0.2% in AP, and 0.2% in RL directions, despite respiratory control. Conclusion: Quantitative analysis of surgical clip displacement reflect respiratory motion, setup errors and postoperative change of intraabdominal organ position. Furthermore, position of clips is distinguished easily in verification images. The identification of the surgical clip position may lead to a significant improvement in the accuracy of upper abdominal radiation therapy.

• The Journal of Advanced Prosthodontics
• /
• v.13 no.4
• /
• pp.246-257
• /
• 2021

PURPOSE. This study aimed to investigate the Vickers Hardness Number (VHN) of light- and dual cured resin cements cured through monolithic zirconia specimens (VITA YZ) of various translucencies: translucent (T); high translucent (HT); super translucent (ST); and extra translucent (XT) at 0, 24, and 48 h after curing. MATERIALS AND METHODS. Four zirconia specimens from each translucency were prepared. Two light-cured resin cements (Variolink N LC; VL and RelyX Veneer; RL) and two dual-cured resin cements (Variolink N DC; VD and RelyX U200; RD) were used. The cement was mixed and loaded in a mold and cured for 20 s through the zirconia specimen. The upper surface of cements was tested for VHN using a microhardness tester at 0, 24, and 48 h after curing. The VHN were analyzed using two-way repeated, Brown-Forsythe ANOVA with Games Howell post-hoc analysis and independent t-tests (P < .05). RESULTS. All cements showed significantly higher VHN from 0 h to 24 h (P < .001). At 48 h, the VHN of light-cured cements were significantly lower when cured under the T groups than under XT groups (P = .001 in VL, P = .014 in RL). At each post curing time of each translucency, VD showed higher VHN than VL (P < .05), and RD also showed higher VHN than RL (P < .05). CONCLUSION. The translucency of zirconia has an effect on the VHN for light-cured resin cements, but has no effect on dual-cured resin cements. Dual-cured resin cement exhibited higher VHN than the light-cured resin cement from the same manufacturer. All resin cements showed significantly higher VHN from 0 h to 24 h.

• Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
• /
• v.29 no.6
• /
• pp.593-599
• /
• 2011

With increasing user applicability of geospatial data, user demand for manifold and accurate information has increased. The usefulness of these services derives from their combination of the advantages of as-built geospatial data in making new content. There is a spatial inconsistency and shape disagreement in fusing heterogeneous data. Conflation, defined as the combining of information from diverse sources so as to reconcile spatial inconsistencies and shape disagreement, is possible solution to the problem. In this research, we developed the technique for removing shape disagreement between aerial image and road map removed spatial inconsistency in advanced research. The process includes four processes: producing of a road candidate image, extraction of vertices, and generation of a graph by connecting the vertices. We could remove the shape disagreement using the extracted road that was derived from finding the road possible path.

• Nuclear Engineering and Technology
• /
• v.56 no.4
• /
• pp.1472-1479
• /
• 2024

Cyber-Physical Energy Systems (CPESs) integrate cyber and hardware components to ensure a reliable and safe physical power production and supply. Renewable Energy Sources (RESs) add uncertainty to energy demand that can be dealt with flexible operation (e.g., load-following) of CPES; at the same time, scenarios that could result in severe consequences due to both component stochastic failures and aging of the cyber system of CPES (commonly overlooked) must be accounted for Operation & Maintenance (O&M) planning. In this paper, we make use of Deep Reinforcement Learning (DRL) to search for the optimal O&M strategy that, not only considers the actual system hardware components health conditions and their Remaining Useful Life (RUL), but also the possible accident scenarios caused by the failures and the aging of the hardware and the cyber components, respectively. The novelty of the work lies in embedding the cyber aging model into the CPES model of production planning and failure process; this model is used to help the RL agent, trained with Proximal Policy Optimization (PPO) and Imitation Learning (IL), finding the proper rejuvenation timing for the cyber system accounting for the uncertainty of the cyber system aging process. An application is provided, with regards to the Advanced Lead-cooled Fast Reactor European Demonstrator (ALFRED).