• Journal of the Society of Naval Architects of Korea
• /
• v.60 no.3
• /
• pp.202-211
• /
• 2023

The profile shops in shipyards produce section steels required for block production of ships. Due to the limitations of shipyard's production capacity, a considerable amount of work is already outsourced. In addition, the need to improve the productivity of the profile shops is growing because the production volume is expected to increase due to the recent boom in the shipbuilding industry. In this study, a scheduling optimization was conducted for a parallel welding line of the profile process, with the aim of minimizing tardiness and the number of set-up changes as objective functions to achieve productivity improvements. In particular, this study applied a dynamic scheduling method to determine the job sequence considering variability of processing time. A Markov decision process model was proposed for the job sequence problem, considering the trade-off relationship between two objective functions. Deep reinforcement learning was also used to learn the optimal scheduling policy. The developed algorithm was evaluated by comparing its performance with priority rules (SSPT, ATCS, MDD, COVERT rule) in test scenarios constructed by the sampling data. As a result, the proposed scheduling algorithms outperformed than the priority rules in terms of set-up ratio, tardiness, and makespan.

• Smart Structures and Systems
• /
• v.31 no.3
• /
• pp.247-257
• /
• 2023

In structural health monitoring of large-scale structures, optimal sensor placement plays an important role because of the high cost of sensors and their supporting instruments, as well as the burden of data transmission and storage. In this study, a vibration sensor placement algorithm based on deep reinforcement learning (DRL) is proposed, which can effectively solve non-convex, high-dimensional, and discrete combinatorial sensor placement optimization problems. An objective function is constructed to estimate the quality of a specific vibration sensor placement scheme according to the modal assurance criterion (MAC). Using this objective function, a DRL-based algorithm is presented to determine the optimal vibration sensor placement scheme. Subsequently, we transform the sensor optimal placement process into a Markov decision process and employ a DRL-based optimization algorithm to maximize the objective function for optimal sensor placement. To illustrate the applicability of the proposed method, two examples are presented: a 10-story braced frame and a sea-crossing bridge model. A comparison study is also performed with a genetic algorithm and particle swarm algorithm. The proposed DRL-based algorithm can effectively solve the discrete combinatorial optimization problem for vibration sensor placements and can produce superior performance compared with the other two existing methods.

• 제어로봇시스템학회:학술대회논문집
• /
• 2003.10a
• /
• pp.174-179
• /
• 2003

Through the simultaneous localization and map building (SLAM) technique, a robot can create maps about its unknown environment while it continuously localizes its position. Grid maps and feature maps have been widely used for SLAM together with application of probability methods and POMDP (partially observed Markov decision process). But this approach based on grid maps suffers from enormous computational burden. Topological maps, however, have drawn more attention these days because they are compact, provide natural interfaces, and are easily applicable to path planning in comparison with grid maps. Some topological SLAM techniques like GVG (generalized Voronoi diagram) were introduced, but it enables the robot to decide only whether the current position is part of GVG branch or not in the GVG algorithm. In this paper, therefore, to overcome these problems, we present a method for updating a global topological map from the local topological maps. These local topological maps are created through a labeled Voronoi diagram algorithm from the local grid map built based on the sensor information at the current robot position. And the nodes of a local topological map can be utilized as the features of the environment because it is robust in light of visibility problem. The geometric information of the feature is applied to the extended Kalman filter and the SLAM in the indoor environment is accomplished. A series of simulations have been conducted using a two-wheeled mobile robot equipped with a laser scanner. It is shown that the proposed scheme can be applied relatively well.

• Journal of the Institute of Convergence Signal Processing
• /
• v.15 no.2
• /
• pp.24-29
• /
• 2014

Digital mammography is the most common technique for the early detection of breast cancer. To diagnose the breast cancer in early stages and treat efficiently, many image enhancement methods have been developed. This paper presents a multi-scale contrast enhancement method in the Laplacian pyramid for the digital mammogram. The proposed method decomposes the image into the contrast measures by the Gaussian and Laplacian pyramid, and the pyramid coefficients of decomposed multi-resolution image are defined as the frequency limited local contrast measures by the ratio of high frequency components and low frequency components. The decomposed pyramid coefficients are modified by the contrast measure for enhancing the contrast, and the final enhanced image is obtained by the composition process of the pyramid using the modified coefficients. The proposed method is compared with other existing methods, and demonstrated to have quantitatively good performance in the contrast measure algorithm.

• Journal of the Korea Institute of Information Security & Cryptology
• /
• v.27 no.6
• /
• pp.1499-1506
• /
• 2017

The use of smartphones and the launch of various apps have increased exponentially, and malicious apps have also increased. Existing app recommendation systems have been limited to operate based on static information analysis such as ratings, comments, and popularity categories of other users who are online. In this paper, we first propose a robust app recommendation system that realistically uses dynamic information of apps actually used in smartphone and considers static information and dynamic information at the same time. In other words, this paper proposes a robust Android app recommendation system by partially reflecting the time of the app, the frequency of use of the app, the interaction between the app and the app, and the number of contact with the Android kernel. As a result of the performance evaluation, the proposed method proved to be a robust and efficient app recommendation system.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.23 no.4
• /
• pp.9-15
• /
• 2023

Existing simple statistics-based recommendation systems rely solely on students' course enrollment history data, making it difficult to identify classes that match students' preferences. To address this issue, this study proposes a personalized major subject recommendation system based on deep reinforcement learning (DRL). This system gauges the similarity between students based on structured data, such as the student's department, grade level, and course history. Based on this information, it recommends the most suitable major subjects by comprehensively considering information about each available major subject and evaluations of the student's courses. We confirmed that this DRL-based recommendation system provides useful insights for university students while selecting their major subjects, and our simulation results indicate that it outperforms conventional statistics-based recommendation systems by approximately 20%. In light of these results, we propose a new system that offers personalized subject recommendations by incorporating students' course evaluations. This system is expected to assist students significantly in finding major subjects that align with their preferences and academic goals.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.23 no.6
• /
• pp.61-67
• /
• 2023

The rise of 5G and the proliferation of smart devices have underscored the significance of multi-access edge computing (MEC). Amidst this trend, interest in effectively processing computation-intensive and latency-sensitive applications has increased. This study investigated a novel task offloading strategy considering the probabilistic MEC environment to address these challenges. Initially, we considered the frequency of dynamic task requests and the unstable conditions of wireless channels to propose a method for minimizing vehicle power consumption and latency. Subsequently, our research delved into a deep reinforcement learning (DRL) based offloading technique, offering a way to achieve equilibrium between local computation and offloading transmission power. We analyzed the power consumption and queuing latency of vehicles using the deep deterministic policy gradient (DDPG) and deep Q-network (DQN) techniques. Finally, we derived and validated the optimal performance enhancement strategy in a vehicle based MEC environment.

• Journal of the Semiconductor & Display Technology
• /
• v.22 no.3
• /
• pp.112-118
• /
• 2023

As technology advances, the need for enhanced preparedness against cyber-attacks becomes an increasingly critical problem. Therefore, it is imperative to consider various circumstances and to prepare for cyber-attack strategic technology. This paper proposes a method to solve network security problems by applying reinforcement learning to cyber-security. In general, traditional static cyber-security methods have difficulty effectively responding to modern dynamic attack patterns. To address this, we implement cyber-attack scenarios such as 'Tiny Alpha' and 'Small Alpha' and evaluate the performance of various reinforcement learning methods using Network Attack Simulator, which is a cyber-attack simulation environment based on the gymnasium (formerly Open AI gym) interface. In addition, we experimented with different RL algorithms such as value-based methods (Q-Learning, Deep-Q-Network, and Double Deep-Q-Network) and policy-based methods (Actor-Critic). As a result, we observed that value-based methods with discrete action spaces consistently outperformed policy-based methods with continuous action spaces, demonstrating a performance difference ranging from a minimum of 20.9% to a maximum of 53.2%. This result shows that the scheme not only suggests opportunities for enhancing cybersecurity strategies, but also indicates potential applications in cyber-security education and system validation across a large number of domains such as military, government, and corporate sectors.

• Journal of Intelligence and Information Systems
• /
• v.25 no.2
• /
• pp.167-194
• /
• 2019

This research starts from the four basic concepts of incentive incompatibility, limited information, myopia and decision variable which are confronted when making decisions in keyword bidding. In order to make these concept concrete, four framework approaches are designed as follows; Strategic approach for the incentive incompatibility, Statistical approach for the limited information, Alternative optimization for myopia, and New model approach for decision variable. The purpose of this research is to propose the statistical optimization model in constructing the portfolio of Sponsored Search Advertising (SSA) in the Sponsor's perspective through empirical tests which can be used in portfolio decision making. Previous research up to date formulates the CTR estimation model using CPC, Rank, Impression, CVR, etc., individually or collectively as the independent variables. However, many of the variables are not controllable in keyword bidding. Only CPC and Rank can be used as decision variables in the bidding system. Classical SSA model is designed on the basic assumption that the CPC is the decision variable and CTR is the response variable. However, this classical model has so many huddles in the estimation of CTR. The main problem is the uncertainty between CPC and Rank. In keyword bid, CPC is continuously fluctuating even at the same Rank. This uncertainty usually raises questions about the credibility of CTR, along with the practical management problems. Sponsors make decisions in keyword bids under the limited information, and the strategic portfolio approach based on statistical models is necessary. In order to solve the problem in Classical SSA model, the New SSA model frame is designed on the basic assumption that Rank is the decision variable. Rank is proposed as the best decision variable in predicting the CTR in many papers. Further, most of the search engine platforms provide the options and algorithms to make it possible to bid with Rank. Sponsors can participate in the keyword bidding with Rank. Therefore, this paper tries to test the validity of this new SSA model and the applicability to construct the optimal portfolio in keyword bidding. Research process is as follows; In order to perform the optimization analysis in constructing the keyword portfolio under the New SSA model, this study proposes the criteria for categorizing the keywords, selects the representing keywords for each category, shows the non-linearity relationship, screens the scenarios for CTR and CPC estimation, selects the best fit model through Goodness-of-Fit (GOF) test, formulates the optimization models, confirms the Spillover effects, and suggests the modified optimization model reflecting Spillover and some strategic recommendations. Tests of Optimization models using these CTR/CPC estimation models are empirically performed with the objective functions of (1) maximizing CTR (CTR optimization model) and of (2) maximizing expected profit reflecting CVR (namely, CVR optimization model). Both of the CTR and CVR optimization test result show that the suggested SSA model confirms the significant improvements and this model is valid in constructing the keyword portfolio using the CTR/CPC estimation models suggested in this study. However, one critical problem is found in the CVR optimization model. Important keywords are excluded from the keyword portfolio due to the myopia of the immediate low profit at present. In order to solve this problem, Markov Chain analysis is carried out and the concept of Core Transit Keyword (CTK) and Expected Opportunity Profit (EOP) are introduced. The Revised CVR Optimization model is proposed and is tested and shows validity in constructing the portfolio. Strategic guidelines and insights are as follows; Brand keywords are usually dominant in almost every aspects of CTR, CVR, the expected profit, etc. Now, it is found that the Generic keywords are the CTK and have the spillover potentials which might increase consumers awareness and lead them to Brand keyword. That's why the Generic keyword should be focused in the keyword bidding. The contribution of the thesis is to propose the novel SSA model based on Rank as decision variable, to propose to manage the keyword portfolio by categories according to the characteristics of keywords, to propose the statistical modelling and managing based on the Rank in constructing the keyword portfolio, and to perform empirical tests and propose a new strategic guidelines to focus on the CTK and to propose the modified CVR optimization objective function reflecting the spillover effect in stead of the previous expected profit models.

• Journal of Intelligence and Information Systems
• /
• v.29 no.2
• /
• pp.101-114
• /
• 2023

In the wartime, aircraft carrying out a mission to strike the enemy deep in the depth are exposed to the risk of being shoot down. As a key combat force in mordern warfare, it takes a lot of time, effot and national budget to train military flight personnel who operate high-tech weapon systems. Therefore, this study studied the path problem of predicting the route of emergency escape from enemy territory to the target point to avoid obstacles, and through this, the possibility of safe recovery of emergency escape military flight personnel was increased. based problem, transforming the problem into a TSP, VRP, and Dijkstra algorithm, and approaching it with an optimization technique. However, if this problem is approached in a network problem, it is difficult to reflect the dynamic factors and uncertainties of the battlefield environment that military flight personnel in distress will face. So, MDP suitable for modeling dynamic environments was applied and studied. In addition, GIS was used to obtain topographic information data, and in the process of designing the reward structure of MDP, topographic information was reflected in more detail so that the model could be more realistic than previous studies. In this study, value iteration algorithms and deterministic methods were used to derive a path that allows the military flight personnel in distress to move to the shortest distance while making the most of the topographical advantages. In addition, it was intended to add the reality of the model by adding actual topographic information and obstacles that the military flight personnel in distress can meet in the process of escape and escape. Through this, it was possible to predict through which route the military flight personnel would escape and escape in the actual situation. The model presented in this study can be applied to various operational situations through redesign of the reward structure. In actual situations, decision support based on scientific techniques that reflect various factors in predicting the escape route of the military flight personnel in distress and conducting combat search and rescue operations will be possible.