• ETRI Journal
• /
• v.40 no.4
• /
• pp.421-434
• /
• 2018

The application of deep neural networks (DNNs) to connect the world with cyber physical systems (CPSs) has attracted much attention. However, DNNs require a large amount of memory and computational cost, which hinders their use in the relatively low-end smart devices that are widely used in CPSs. In this paper, we aim to determine whether DNNs can be efficiently deployed and operated in low-end smart devices. To do this, we develop a method to reduce the memory requirement of DNNs and increase the inference speed, while maintaining the performance (for example, accuracy) close to the original level. The parameters of DNNs are decomposed using a hybrid of canonical polyadic-singular value decomposition, approximated using a tensor power method, and fine-tuned by performing iterative one-shot hybrid fine-tuning to recover from a decreased accuracy. In this study, we evaluate our method on frequently used networks. We also present results from extensive experiments on the effects of several fine-tuning methods, the importance of iterative fine-tuning, and decomposition techniques. We demonstrate the effectiveness of the proposed method by deploying compressed networks in smartphones.

• The Journal of the Acoustical Society of Korea
• /
• v.39 no.6
• /
• pp.505-514
• /
• 2020

Since a large amount of training data is typically needed to train Deep Neural Networks (DNNs), a parallel training approach is required to train the DNNs. The Stochastic Gradient Descent (SGD) algorithm is one of the most widely used methods to train the DNNs. However, since the SGD is an inherently sequential process, it requires some sort of approximation schemes to parallelize the SGD algorithm. In this paper, we review various efforts on parallelizing the SGD algorithm, and analyze the computational overhead, communication overhead, and the effects of the approximations.

• Journal of Sensor Science and Technology
• /
• v.30 no.1
• /
• pp.20-24
• /
• 2021

The electrocardiogram (ECG) is one of the most extensively employed signals used to diagnose and predict cardiovascular diseases (CVDs). In recent years, several deep learning (DL) models have been proposed to improve detection accuracy. Among these, deep neural networks (DNNs) are the most popular, wherein the features are extracted automatically. Despite the increment in classification accuracy, DL models require exorbitant computational resources and power. This causes the mapping of DNNs to be slow; in addition, the mapping is challenging for a wearable device. Embedded systems have constrained power and memory resources. Therefore full-precision DNNs are not easily deployable on devices. To make the neural network faster and more power-efficient, spiking neural networks (SNNs) have been introduced for fewer operations and less complex hardware resources. However, the conventional SNN has low accuracy and high computational cost. Therefore, this paper proposes a new binarized SNN which modifies the synaptic weights of SNN constraining it to be binary (+1 and -1). In the simulation results, this paper compares the DL models and SNNs and evaluates which model is optimal for ECG classification. Although there is a slight compromise in accuracy, the latter proves to be energy-efficient.

• Journal of Broadcast Engineering
• /
• v.26 no.7
• /
• pp.862-867
• /
• 2021

Although Deep Neural Networks (DNNs) have shown remarkable performance in various artificial intelligence fields, it is well known that DNNs are vulnerable to adversarial attacks. Since adversarial attacks are implemented by adding perturbations onto benign examples, increasing the sparsity of DNNs minimizes the propagation of errors to high-level layers. In this paper, unlike the traditional pruning scheme removing low magnitude weights, we eliminate high magnitude weights that are usually considered high absolute values, named 'reverse pruning' to ensure robustness. By conducting both theoretical and experimental analyses, we observe that reverse pruning ensures the robustness of DNNs. Experimental results show that our reverse pruning outperforms previous work with 29.01% in Top-1 accuracy on perturbed CIFAR-10. However, reverse pruning does not guarantee benign samples. To relax this problem, we further conducted experiments by adding a regularization term for the high magnitude weights. With adding the regularization term, we also applied conventional pruning to ensure the robustness of DNNs.

• Journal of the Semiconductor & Display Technology
• /
• v.21 no.1
• /
• pp.75-78
• /
• 2022

Deep Neural Networks (DNN) has become an essential data processing architecture for the implementation of multiple computer vision tasks. Recently, DNN-based algorithms achieve much higher recognition accuracy than traditional algorithms based on shallow learning. However, training and inference DNNs require huge computational capabilities than daily usage purposes of computers. Moreover, with increased size and depth of DNNs, CPUs may be unsatisfactory since they use serial processing by default. GPUs are the solution that come up with greater speed compared to CPUs because of their Parallel Processing/Computation nature. In this paper, we analyze the inference time complexity of DNNs using well-known computer vision library, OpenCV. We measure and analyze inference time complexity for three cases, CPU, GPU-Float32, and GPU-Float16.

• Current Optics and Photonics
• /
• v.8 no.1
• /
• pp.80-85
• /
• 2024

Numerical values of the Auger coefficient and the free carrier absorption (FCA) coefficient are extracted by applying deep neural networks (DNNs) to the L-I characteristics of 850 nm GaAs/AlGaAs laser diodes. Two elemental DNNs are used to extract each coefficient sequentially. The fidelity of the extracted values is established through meticulous correlation of L-I characteristics bridging the realms of simulations and measurements. The methodology presented in this paper offers a way to accurately extract the Auger and FCA coefficients, which were traditionally treated as fitting parameters. It is anticipated that this approach will be applicable to other types of opto-electronic devices as well.

• Journal of Biomedical Engineering Research
• /
• v.41 no.2
• /
• pp.107-119
• /
• 2020

The deep neural networks (DNN) that can replicate the behavior of the human expert who recognizes the characteristics of ECG waveform have been developed and studied to analyze ECG. However, although the existing DNNs can not provide the explanations for their decisions, those trials have attempted to determine whether patients have certain diseases or not and those decisions could not be accepted because of the absence of relating theoretical basis. In addition, these DNNs required a lot of training data to obtain sufficient accuracy in spite of the difficulty in the acquisition of relating clinical data. In this study, a small-sized continuous data processing DNN (C-DNN) was suggested to determine the simple characteristics of ECG wave that were not required additional explanations about its decisions and the C-DNN can be easily trained with small training data. Although it can analyze small input data that was selected in narrow region on whole ECG, it can continuously scan all ECG data and find important points such as start and end points of P, QRS and T waves within a short time. The star and end points of ECG waves determined by the C-DNNs were compared with the results performed by human experts to estimate the accuracies of the C-DNNs. The C-DNN has 150 inputs, 51 outputs, two hidden layers and one output layer. To find the start and end points, two C-DNNs were trained through deep learning technology and applied to a parameter acquisition algorithms. 12 lead ECG data measured in four patients and obtained through PhysioNet was processed to make training data by human experts. The accuracy of the C-DNNs were evaluated with extra data that were not used at deep learning by comparing the results between C-DNNs and human experts. The averages of the time differences between the C-DNNs and experts were 0.1 msec and 13.5 msec respectively and those standard deviations were 17.6 msec and 15.7 msec. The final step combining the results of C-DNN through the waveforms of 12 leads was successfully determined all 33 waves without error that the time differences of human experts decision were over 20 msec. The reliable decision of the ECG wave's start and end points benefits the acquisition of accurate ECG parameters such as the wave lengths, amplitudes and intervals of P, QRS and T waves.

• Annual Conference on Human and Language Technology
• /
• 2017.10a
• /
• pp.238-241
• /
• 2017

본 논문에서는 구글 학술 검색 기반의 데이터를 이용하여 질병과 폐질환과 관련된 바이오마커 단어의 유사도를 계산하는 방법을 제안한다. 질병과 바이오마커의 유사도를 계산할 때, 각 단어의 구글 학술 검색의 검색 결과를 이용하였다. 이를 통해 폐질환 관련 바이오마커와 다른 질병간의 관계를 파악하고자 하며, 의료 전문가에게 폐질환 관련 바이오마커와 다른 질병간의 새로운 관계를 제시하고자 한다. 이러한 데이터를 이용하여 계산한 결과, Wor2Vec의 결과를 이용한 코사인 유사도의 결과와 상관 계수가 약 0.64로 상당히 높은 상관 관계를 확인할 수 있었다. 따라서 이 방법을 통해 질병과 바이오마커의 관계를 파악하고자 하였다. 또한 Word2Vec을 이용한 질병과 바이오마커 단어의 벡터 값과 단어 유사도 계산 방법의 결과를 이용한 Deep Neural Networks (DNNs) 모델을 구축하고자 하며, 이를 통해 자동적으로 유사도를 분석하고자 하였다.

• 한국어정보학회:학술대회논문집
• /
• 2017.10a
• /
• pp.238-241
• /
• 2017

본 논문에서는 구글 학술 검색 기반의 데이터를 이용하여 질병과 폐질환과 관련된 바이오마커 단어의 유사도를 계산하는 방법을 제안한다. 질병과 바이오마커의 유사도를 계산할 때, 각 단어의 구글 학술 검색의 검색 결과를 이용하였다. 이를 통해 폐질환 관련 바이오마커와 다른 질병간의 관계를 파악하고자 히며, 의료 전문가에게 폐질환 관련 바이오마커와 다른 질병간의 새로운 관계를 제시하고자 한다. 이러한 데이터를 이용하여 계산한 결과, Wor2Vec의 결과를 이용한 코사인 유사도의 결과와 상관 계수가 약 0.64로 상당히 높은 상관 관계를 확인할 수 있었다. 따라서 이 방법을 통해 질병과 바이오마커의 관계를 파악하고자 하였다. 또한 Word2Vec을 이용한 질병과 바이오마커 단어의 벡터 값과 단어 유사도 계산 방법의 결과를 이용한 Deep Neural Networks (DNNs) 모델을 구축하고자 하며, 이를 통해 자동적으로 유사도를 분석하고자 하였다.

• Smart Structures and Systems
• /
• v.29 no.1
• /
• pp.141-151
• /
• 2022

In recent years, the industry and research communities have focused on developing autonomous crack inspection approaches, which mainly include image acquisition and crack detection. In these approaches, mobile devices such as cameras, drones or smartphones are utilized as sensing platforms to acquire structural images, and the deep learning (DL)-based methods are being developed as important crack detection approaches. However, the process of image acquisition and collection is time-consuming, which delays the inspection. Also, the present mobile devices such as smartphones can be not only a sensing platform but also a computing platform that can be embedded with deep neural networks (DNNs) to conduct on-site crack detection. Due to the limited computing resources of mobile devices, the size of the DNNs should be reduced to improve the computational efficiency. In this study, an architecture called pruned crack recognition network (PCR-Net) was developed for the detection of structural cracks. A dataset containing 11000 images was established based on the raw images from bridge inspections. A pruning method was introduced to reduce the size of the base architecture for the optimization of the model size. Comparative studies were conducted with image processing techniques (IPTs) and other DNNs for the evaluation of the performance of the proposed PCR-Net. Furthermore, a modularly designed framework that integrated the PCR-Net was developed to realize a DL-based crack detection application for smartphones. Finally, on-site crack detection experiments were carried out to validate the performance of the developed system of smartphone-based detection of structural cracks.