• Journal of Korea Water Resources Association
• /
• v.55 no.11
• /
• pp.903-911
• /
• 2022

Groundwater, one of the resources for supplying water, fluctuates in water level due to various natural factors. Recently, research has been conducted to predict fluctuations in groundwater levels using Artificial Neural Network (ANN). Previously, among operators in ANN, Gradient Descent (GD)-based Optimizers were used as Optimizer that affect learning. GD-based Optimizers have disadvantages of initial correlation dependence and absence of solution comparison and storage structure. This study developed Gradient Descent combined with Harmony Search (GDHS), a new Optimizer that combined GD and Harmony Search (HS) to improve the shortcomings of GD-based Optimizers. To evaluate the performance of GDHS, groundwater level at Icheon Yullhyeon observation station were learned and predicted using Multi Layer Perceptron (MLP). Mean Squared Error (MSE) and Mean Absolute Error (MAE) were used to compare the performance of MLP using GD and GDHS. Comparing the learning results, GDHS had lower maximum, minimum, average and Standard Deviation (SD) of MSE than GD. Comparing the prediction results, GDHS was evaluated to have a lower error in all of the evaluation index than GD.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.23 no.11
• /
• pp.1357-1363
• /
• 2019

In this paper, we compares the performance of the gredient descent optimizers of the Faster Region-based Convolutional Neural Network (R-CNN) model for the chromosome object detection in digital images composed of human metaphase chromosomes. In faster R-CNN, the gradient descent optimizer is used to minimize the objective function of the region proposal network (RPN) module and the classification score and bounding box regression blocks. The gradient descent optimizer. Through performance comparisons among these four gradient descent optimizers in our experiments, we found that the Adamax optimizer could achieve the mean average precision (mAP) of about 52% when considering faster R-CNN with a base network, VGG16. In case of faster R-CNN with a base network, ResNet50, the Adadelta optimizer could achieve the mAP of about 58%.

• Proceedings of the Korea Water Resources Association Conference
• /
• 2022.05a
• /
• pp.186-186
• /
• 2022

강수 및 침투 등으로 발생하는 지하수위의 변동을 예측하는 것은 지하수 자원의 활용 및 관리에 필수적이다. 지하수위의 변동은 지하수 자원의 활용 및 관리뿐만이 아닌 홍수 발생과 지반의 응력상태 등에 직접적인 영향을 미치기 때문에 정확한 예측이 필요하다. 본 연구는 인공신경망 중 다층퍼셉트론(Multi Layer Perceptron, MLP)을 이용한 지하수위 예측성능 향상을 위해 MLP의 구조 중 Optimizer를 개량하였다. MLP는 입력자료와 출력자료간 최적의 상관관계(가중치 및 편향)를 찾는 Optimizer와 출력되는 값을 결정하는 활성화 함수의 연산을 반복하여 학습한다. 특히 Optimizer는 신경망의 출력값과 관측값의 오차가 최소가 되는 상관관계를 찾는 연산자로써 MLP의 학습 및 예측성능에 직접적인 영향을 미친다. 기존의 Optimizer는 경사하강법(Gradient Descent, GD)을 기반으로 하는 Optimizer를 사용했다. 하지만 기존의 Optimizer는 미분을 이용하여 상관관계를 찾기 때문에 지역탐색 위주로 진행되며 기존에 생성된 상관관계를 저장하는 구조가 없어 지역 최적해로 수렴할 가능성이 있다는 단점이 있다. 본 연구에서는 기존 Optimizer의 단점을 개선하기 위해 지역탐색과 전역탐색을 동시에 고려할 수 있으며 기존의 해를 저장하는 구조가 있는 메타휴리스틱 최적화 알고리즘을 이용하였다. 메타휴리스틱 최적화 알고리즘 중 구조가 간단한 화음탐색법(Harmony Search, HS)과 GD의 결합모형(HS-GD)을 MLP의 Optimizer로 사용하여 기존 Optimizer의 단점을 개선하였다. HS-GD를 이용한 MLP의 성능검토를 위해 이천시 지하수위 예측을 실시하였으며 예측 결과를 기존의 Optimizer를 이용한 MLP 및 HS를 이용한 MLP의 예측결과와 비교하였다.

• Journal of Integrative Natural Science
• /
• v.11 no.4
• /
• pp.204-208
• /
• 2018

Comparison of different optimizer performance in photovoltaic power modeling using artificial neural deep learning techniques is described in this paper. Six different deep learning optimizers are tested for Long-Short-Term Memory networks in this study. The optimizers are namely Adam, Stochastic Gradient Descent, Root Mean Square Propagation, Adaptive Gradient, and some variants such as Adamax and Nadam. For comparing the optimization techniques, high and low fluctuated photovoltaic power output are examined and the power output is real data obtained from the site at Mokpo university. Using Python Keras version, we have developed the prediction program for the performance evaluation of the optimizations. The prediction error results of each optimizer in both high and low power cases shows that the Adam has better performance compared to the other optimizers.

• Journal of Korea Water Resources Association
• /
• v.56 no.1
• /
• pp.63-74
• /
• 2023

High-reliability prediction of dam inflow is necessary for efficient dam operation. Recently, studies were conducted to predict the inflow of dams using Multi Layer Perceptron (MLP). Existing studies used the Gradient Descent (GD)-based optimizer as the optimizer among MLP operators to find the optimal correlation between data. However, the GD-based optimizers have disadvantages in that the prediction performance is deteriorated due to the possibility of convergence to the local optimal value and the absence of storage space. This study improved the shortcomings of the GD-based optimizer by developing Adaptive moments combined with Improved Harmony Search (AdamIHS), which combines Adaptive moments among GD-based optimizers and Improved Harmony Search (IHS). In order to evaluate the learning and prediction performance of MLP using AdamIHS, Daecheong Dam inflow was learned and predicted and compared with the learning and prediction performance of MLP using GD-based optimizer. Comparing the learning results, the Mean Squared Error (MSE) of MLP, which is 5 hidden layers using AdamIHS, was the lowest at 11,577. Comparing the prediction results, the average MSE of MLP, which is one hidden layer using AdamIHS, was the lowest at 413,262. Using AdamIHS developed in this study, it will be possible to show improved prediction performance in various fields.

• Korean Journal of Remote Sensing
• /
• v.38 no.6_1
• /
• pp.1329-1341
• /
• 2022

Destroying the marine environment and marine ecosystem and causing marine accidents, marine debris is generated every year, and among them, submerged marine debris is difficult to identify and collect because it is on the seabed. Therefore, deep-learning-based semantic segmentation was experimented on waste fish nets and waste ropes using underwater images to identify efficient collection and distribution. For segmentation, a high-resolution network (HRNet), a state-of-the-art deep learning technique, was used, and the performance of each optimizer was compared. In the segmentation result fish net, F1 score=(86.46%, 86.20%, 85.29%), IoU=(76.15%, 75.74%, 74.36%), For the rope F1 score=(80.49%, 80.48%, 77.86%), IoU=(67.35%, 67.33%, 63.75%) in the order of adaptive moment estimation (Adam), Momentum, and stochastic gradient descent (SGD). Adam's results were the highest in both fish net and rope. Through the research results, the evaluation of segmentation performance for each optimizer and the possibility of segmentation of marine debris in the latest deep learning technique were confirmed. Accordingly, it is judged that by applying the latest deep learning technique to the identification of submerged marine debris through underwater images, it will be helpful in estimating the distribution of marine sedimentation debris through more accurate and efficient identification than identification through the naked eye.

• International Journal of Computer Science & Network Security
• /
• v.23 no.10
• /
• pp.115-128
• /
• 2023

Deep learning has been incorporating various optimization techniques motivated by new pragmatic optimizing algorithm advancements and their usage has a central role in Machine learning. In recent past, new avatars of various optimizers are being put into practice and their suitability and applicability has been reported on various domains. The resurgence of novelty starts from Stochastic Gradient Descent to convex and non-convex and derivative-free approaches. In the contemporary of these horizons of optimizers, choosing a best-fit or appropriate optimizer is an important consideration in deep learning theme as these working-horse engines determines the final performance predicted by the model. Moreover with increasing number of deep layers tantamount higher complexity with hyper-parameter tuning and consequently need to delve for a befitting optimizer. We empirically examine most popular and widely used optimizers on various data sets and networks-like MNIST and GAN plus others. The pragmatic comparison focuses on their similarities, differences and possibilities of their suitability for a given application. Additionally, the recent optimizer variants are highlighted with their subtlety. The article emphasizes on their critical role and pinpoints buttress options while choosing among them.

• Journal of the Korean Society of Radiology
• /
• v.16 no.1
• /
• pp.25-34
• /
• 2022

Brain computed tomography (CT) is useful for brain lesion diagnosis, such as brain hemorrhage, due to non-invasive methodology, 3-dimensional image provision, low radiation dose. However, there has been numerous misdiagnosis owing to a lack of radiologist and heavy workload. Recently, object detection technologies based on artificial intelligence have been developed in order to overcome the limitations of traditional diagnosis. In this study, the applicability of a deep learning-based YOLOv5s model was evaluated for brain hemorrhage detection using brain CT images. Also, the effect of hyperparameters in the trained YOLOv5s model was analyzed. The YOLOv5s model consisted of backbone, neck and output modules. The trained model was able to detect a region of brain hemorrhage and provide the information of the region. The YOLOv5s model was trained with various activation functions, optimizer functions, loss functions and epochs, and the performance of the trained model was evaluated in terms of brain hemorrhage detection accuracy and training time. The results showed that the trained YOLOv5s model is able to provide a bounding box for a region of brain hemorrhage and the accuracy of the corresponding box. The performance of the YOLOv5s model was improved by using the mish activation function, the stochastic gradient descent (SGD) optimizer function and the completed intersection over union (CIoU) loss function. Also, the accuracy and training time of the YOLOv5s model increased with the number of epochs. Therefore, the YOLOv5s model is suitable for brain hemorrhage detection using brain CT images, and the performance of the model can be maximized by using appropriate hyperparameters.

• Journal of Korea Water Resources Association
• /
• v.57 no.2
• /
• pp.73-85
• /
• 2024

Predicting water quality of rivers and reservoirs is necessary for the management of water resources. Artificial Neural Networks (ANNs) have been used in many studies to predict water quality with high accuracy. Previous studies have used Gradient Descent (GD)-based optimizers as an optimizer, an operator of ANN that searches parameters. However, GD-based optimizers have the disadvantages of the possibility of local optimal convergence and absence of a solution storage and comparison structure. This study developed improved optimizers to overcome the disadvantages of GD-based optimizers. Proposed optimizers are optimizers that combine adaptive moments (Adam) and Nesterov-accelerated adaptive moments (Nadam), which have low learning errors among GD-based optimizers, with Harmony Search (HS) or Novel Self-adaptive Harmony Search (NSHS). To evaluate the performance of Long Short-Term Memory (LSTM) using improved optimizers, the water quality data from the Dasan water quality monitoring station were used for training and prediction. Comparing the learning results, Mean Squared Error (MSE) of LSTM using Nadam combined with NSHS (NadamNSHS) was the lowest at 0.002921. In addition, the prediction rankings according to MSE and R² for the four water quality indices for each optimizer were compared. Comparing the average of ranking for each optimizer, it was confirmed that LSTM using NadamNSHS was the highest at 2.25.

• Proceedings of the Korean Society for Agricultural Machinery Conference
• /
• 2017.04a
• /
• pp.162-162
• /
• 2017

구글에서 공개한 Tensorflow를 이용한 여러 학문 분야의 연구가 활발하다. 농업 시설환경을 대상으로 한 빅데이터의 축적이 증가함과 아울러 실효적인 정보 획득을 위한 각종 데이터 분석 및 마이닝 기법에 대한 연구 또한 활발한 상황이다. 한편, 타 분야의 성공적인 심층학습기법 응용사례에 비하여 농업 분야에서의 응용은 초기 성장 단계라 할 수 있다. 이는 농업 현장에서 취득한 정보의 난해성 및 완성도 높은 생육/환경 모델링 정보의 부재로 실효적인 전과정 처리 기술 도출에 소요되는 시간, 비용, 연구 환경이 상대적으로 부족하기 때문일 것이다. 특히, 센서 기반 데이터 취득 기술 증가에 따라 비약적으로 방대해진 수집 데이터를 시간 복잡도가 높은 심층 학습 모델링 연산에 기계적으로 단순 적용할 경우 시간 효율적인 측면에서 성공적인 결과 도출에 애로가 있을 것이다. 매우 높은 시간 복잡도를 해결하기 위하여 제시된 하드웨어 가속 기능의 경우 일부 개발환경에 국한이 되어 있다. 일례로, 구글의 Tensorflow는 오픈소스 기반 병렬 클러스터링 기술인 MPICH를 지원하는 알고리즘을 공개하지 않고 있다. 따라서, 본 연구에서는 심층학습 기법 연구에 있어서, 예상 가능한 다양한 자원을 활용하여 최대한 연산의 결과를 빨리 도출할 수 있는 하드웨어적인 접근 방법을 모색하였다. 호스트에서 수행하는 일방적인 학습 알고리즘과 달리 이기종간 심층 학습이 가능하기 위해선 우선, NFS(Network File System)를 이용하여 데이터 계층이 상호 연결이 되어야 한다. 이를 위해서 고속 네트워크를 기반으로 한 NFS의 이용이 필수적이다. 둘째로 제한된 자원의 한계를 극복하기 위한 메모 공유 라이브러리가 필요하다. 셋째로 이기종간 프로세서에 최적화된 병렬 처리용 컴파일러를 이용해야 한다. 가장 중요한 부분은 이기종간의 처리 능력에 따른 작업을 고르게 분배할 수 있는 작업 스케쥴링이 수행되어야 하며, 이는 처리하고자 하는 데이터의 형태에 따라 매우 가변적이므로 해당 데이터 도메인에 대한 엄밀한 사전 벤치마킹이 수행되어야 한다. 이러한 요구조건을 대부분 충족하는 Open-CL ver1.2(https://www.khronos.org/opencl/)를 이용하였다. 최신의 Open-CL 버전은 2.2이나 본 연구를 위하여 준비한 4가지 이기종 시스템에서 모두 공통적으로 지원하는 버전은 1.2이다. 실험적으로 선정된 4가지 이기종 시스템은 1) Windows 10 Pro, 2) Linux-Ubuntu 16.04.4 LTS-x86_64, 3) MAC OS X 10.11 4) Linux-Ubuntu 16.04.4 LTS-ARM Cortext-A15 이다. 비교 분석을 위하여 NVIDIA 사에서 제공하는 Pascal Titan X 2식을 SLI로 구성한 시스템을 준비하였다. 개별 시스템에서 별도로 컴파일 된 바이너리의 이름을 통일하고, 개별 시스템의 코어수를 동일하게 균등 배분하여 100 Hz의 데이터로 입력이 되는 온도 정보와 조도 정보를 입력으로 하고 이를 습도정보에 Linear Gradient Descent Optimizer를 이용하여 Epoch 10,000회의 학습을 수행하였다. 4종의 이기종에서 총 32개의 코어를 이용한 학습에서 17초 내외로 연산 수행을 마쳤으나, 비교 시스템에서는 11초 내외로 연산을 마치는 결과가 나왔다. 기보유 하드웨어의 적절한 활용이 가능한 심층학습 기법에 대한 연구를 지속할 것이다