• Electronics and Telecommunications Trends
• /
• v.38 no.5
• /
• pp.12-22
• /
• 2023

The development of neural networks is evolving towards the adoption of transformer structures with attention modules. Hence, active research focused on extending the concept of lightweight neural network algorithms and hardware acceleration is being conducted for the transition from conventional convolutional neural networks to transformer-based networks. We present a survey of state-of-the-art research on lightweight neural network algorithms and hardware architectures to reduce memory usage and accelerate both inference and training. To describe the corresponding trends, we review recent studies on token pruning, quantization, and architecture tuning for the vision transformer. In addition, we present a hardware architecture that incorporates lightweight algorithms into artificial intelligence processors to accelerate processing.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.8 no.6
• /
• pp.129-135
• /
• 1998

In this paper, the recognition rate of isolated digits has been improved using the multiple neural networks combined with chaotic recurrent neural networks and MLP. Generally, the recognition rate has been increased from 1.2% to 2.5%. The experiments tell that the recognition rate is increased because MLP and CRNN(chaotic recurrent neural network) compensate for each other. Besides this, the chaotic dynamic properties have helped more in speech recognition. The best recognition rate is when the algorithm combined with MLP and chaotic multiple recurrent neural network has been used. However, in the respect of simple algorithm and reliability, the multiple neural networks combined with MLP and chaotic single recurrent neural networks have better properties. Largely, MLP has very good recognition rate in korean digits "il", "oh", while the chaotic recurrent neural network has best recognition in "young", "sam", "chil".

• Proceedings of the Korean Institute of Intelligent Systems Conference
• /
• 2000.05a
• /
• pp.245-248
• /
• 2000

In this paper we improve the performance of autonomous mobile robot by induction of reinforcement learning concept. Generally, the system used in this paper is divided into two part. Namely, one is neural-fuzzy and the other is dynamic recurrent neural networks. Neural-fuzzy determines the next action of robot. Also, the neural-fuzzy is determined to optimal action internal reinforcement from dynamic recurrent neural network. Dynamic recurrent neural network evaluated to determine action of neural-fuzzy by external reinforcement signal from environment, Besides, dynamic recurrent neural network weight determined to internal reinforcement signal value is evolved by genetic algorithms. The architecture of propose system is applied to the computer simulations on controlling autonomous mobile robot.

• 제어로봇시스템학회:학술대회논문집
• /
• 1997.10a
• /
• pp.708-711
• /
• 1997

This paper proposes the hybrid algorithm for the optimization of the structure and parameters of the fuzzy neural networks by genetic algorithms (GA) to improve the behaviour and the design of fuzzy neural networks. Fuzzy neural networks have a distinguishing feature in that they can possess the advantage of both neural networks and fuzzy systems. In this way, we can bring the low-level learning and computational power of neural networks into fuzzy systems and also high-level, human like IF-THEN rule thinking and reasoning of fuzzy systems into neural networks. As a result, there are many research works concerning the optimization of the structure and parameters of fuzzy neural networks. In this paper, we propose the hybrid algorithm that can optimize both the structure and parameters of fuzzy neural networks. Numerical example is provided to show the advantages of the proposed method.

• International Journal of Computer Science & Network Security
• /
• v.23 no.7
• /
• pp.39-48
• /
• 2023

Breast cancer screening makes extensive utilization of mammography. Even so, there has been a lot of debate with regards to this application's starting age as well as screening interval. The deep learning technique of transfer learning is employed for transferring the knowledge learnt from the source tasks to the target tasks. For the resolution of real-world problems, deep neural networks have demonstrated superior performance in comparison with the standard machine learning algorithms. The architecture of the deep neural networks has to be defined by taking into account the problem domain knowledge. Normally, this technique will consume a lot of time as well as computational resources. This work evaluated the efficacy of the deep learning neural network like Visual Geometry Group Network (VGG Net) Residual Network (Res Net), as well as inception network for classifying the mammograms. This work proposed optimization of ResNet with Teaching Learning Based Optimization (TLBO) algorithm's in order to predict breast cancers by means of mammogram images. The proposed TLBO-ResNet, an optimized ResNet with faster convergence ability when compared with other evolutionary methods for mammogram classification.

• Journal of the Korean Geotechnical Society
• /
• v.20 no.8
• /
• pp.77-87
• /
• 2004

In this paper a back-propagation neural network model is developed to estimate the preconsolidation pressure of Korean soft soils based on 176 oedometer tests and 63 piezocone test results, which were compiled from 11 sites - western and southern parts of Korea. Only 147 data were used for the training of the neural network and 29 data, which were not used during the training phase, were used for the verification of trained network. Empirical and theoretical models were compared with the developed neural network model. A simple 4-4-9-1 multi-layered neural network has been developed. The cone tip resistance $q_T$ penetration pore pressure $u_2$, total overburden pressure $\sigma_{vo}$ and effective overburden pressure $\sigma'_{vo}$ were selected as input variables. The developed neural network model was validated by comparing the prediction results of the proposed neural network model for the new data which were not used for the training of the model with the measured preconsolidation pressures. It can also predict more precise and reliable preconsolidation pressures than the analytical and empirical model. Furthermore, it can be carefully concluded that neural network model can be used as a generalized model for prediction of preconsolidation pressure throughout Korea since developed model shows good performance for the new data which were not used in both training and testing data.

• Journal of the Korean Society of Marine Environment & Safety
• /
• v.27 no.1
• /
• pp.13-21
• /
• 2021

The purpose of this study is to develop the ship valuation model by utilizing the neural network model. The target of the valuation was secondhand VLCC. The variables were set as major factors inducing changes in the value of ship through prior research, and the corresponding data were collected on a monthly basis from January 2000 to August 2020. To determine the stability of subsequent variables, a multi-collinearity test was carried out and finally the research structure was designed by selecting six independent variables and one dependent variable. Based on this structure, a total of nine simulation models were designed using linear regression, neural network regression, and random forest algorithm. In addition, the accuracy of the evaluation results are improved through comparative verification between each model. As a result of the evaluation, it was found that the most accurate when the neural network regression model, which consist of a hidden layer composed of two layers, was simulated through comparison with actual VLCC values. The possible implications of this study first, creative research in terms of applying neural network model to ship valuation; this deviates from the existing formalized evaluation techniques. Second, the objectivity of research results was enhanced from a dynamic perspective by analyzing and predicting the factors of changes in the shipping. market.

• Journal of information and communication convergence engineering
• /
• v.21 no.1
• /
• pp.9-16
• /
• 2023

The natural language on social network platforms has a certain front-to-back dependency in structure, and the direct conversion of Chinese text into a vector makes the dimensionality very high, thereby resulting in the low accuracy of existing text classification methods. To this end, this study establishes a deep learning model that combines a big data ultra-deep convolutional neural network (UDCNN) and long short-term memory network (LSTM). The deep structure of UDCNN is used to extract the features of text vector classification. The LSTM stores historical information to extract the context dependency of long texts, and word embedding is introduced to convert the text into low-dimensional vectors. Experiments are conducted on the social network platforms Sogou corpus and the University HowNet Chinese corpus. The research results show that compared with CNN + rand, LSTM, and other models, the neural network deep learning hybrid model can effectively improve the accuracy of text classification.

• 제어로봇시스템학회:학술대회논문집
• /
• 1997.10a
• /
• pp.446-449
• /
• 1997

We propose a control law in discrete time domain of the bilateral feedback teleoperation system using neural network and the reference model type of adaptive control. Different from traditional teleoperation systems, the transmission time delay irregularly changes. The proposed control method controls master and slave systems through identification of master and slave models using neural networks.

• Proceedings of the IEEK Conference
• /
• 1999.11a
• /
• pp.1015-1018
• /
• 1999

This paper describes a vector quantization for speech signal coding using neural networks. We processed speech signal using LPC method that extracts speech signal feature, and speech signal feature is quantized using competitive neural network kohonen self-organization feature map.