• ETRI Journal
• /
• 제40권4호
• /
• pp.421-434
• /
• 2018

The application of deep neural networks (DNNs) to connect the world with cyber physical systems (CPSs) has attracted much attention. However, DNNs require a large amount of memory and computational cost, which hinders their use in the relatively low-end smart devices that are widely used in CPSs. In this paper, we aim to determine whether DNNs can be efficiently deployed and operated in low-end smart devices. To do this, we develop a method to reduce the memory requirement of DNNs and increase the inference speed, while maintaining the performance (for example, accuracy) close to the original level. The parameters of DNNs are decomposed using a hybrid of canonical polyadic-singular value decomposition, approximated using a tensor power method, and fine-tuned by performing iterative one-shot hybrid fine-tuning to recover from a decreased accuracy. In this study, we evaluate our method on frequently used networks. We also present results from extensive experiments on the effects of several fine-tuning methods, the importance of iterative fine-tuning, and decomposition techniques. We demonstrate the effectiveness of the proposed method by deploying compressed networks in smartphones.

• 한국음향학회지
• /
• 제39권6호
• /
• pp.505-514
• /
• 2020

심층 신경망(Deep Neural Network, DNN) 모델을 대량의 학습 데이터로 학습시키기 위해서는 많은 시간이 소요되기 때문에 병렬 학습 방법이 필요하다. DNN의 학습에는 일반적으로 Stochastic Gradient Descent(SGD) 방법이 사용되는데, SGD는 근본적으로 순차적인 처리가 필요하므로 병렬화하기 위해서는 다양한 근사(approximation) 방법을 적용하게 된다. 본 논문에서는 기존의 DNN 병렬 학습 알고리즘들을 소개하고 연산량, 통신량, 근사 방법 등을 분석한다.

• 센서학회지
• /
• 제30권1호
• /
• pp.20-24
• /
• 2021

The electrocardiogram (ECG) is one of the most extensively employed signals used to diagnose and predict cardiovascular diseases (CVDs). In recent years, several deep learning (DL) models have been proposed to improve detection accuracy. Among these, deep neural networks (DNNs) are the most popular, wherein the features are extracted automatically. Despite the increment in classification accuracy, DL models require exorbitant computational resources and power. This causes the mapping of DNNs to be slow; in addition, the mapping is challenging for a wearable device. Embedded systems have constrained power and memory resources. Therefore full-precision DNNs are not easily deployable on devices. To make the neural network faster and more power-efficient, spiking neural networks (SNNs) have been introduced for fewer operations and less complex hardware resources. However, the conventional SNN has low accuracy and high computational cost. Therefore, this paper proposes a new binarized SNN which modifies the synaptic weights of SNN constraining it to be binary (+1 and -1). In the simulation results, this paper compares the DL models and SNNs and evaluates which model is optimal for ECG classification. Although there is a slight compromise in accuracy, the latter proves to be energy-efficient.

• 방송공학회논문지
• /
• 제26권7호
• /
• pp.862-867
• /
• 2021

Although Deep Neural Networks (DNNs) have shown remarkable performance in various artificial intelligence fields, it is well known that DNNs are vulnerable to adversarial attacks. Since adversarial attacks are implemented by adding perturbations onto benign examples, increasing the sparsity of DNNs minimizes the propagation of errors to high-level layers. In this paper, unlike the traditional pruning scheme removing low magnitude weights, we eliminate high magnitude weights that are usually considered high absolute values, named 'reverse pruning' to ensure robustness. By conducting both theoretical and experimental analyses, we observe that reverse pruning ensures the robustness of DNNs. Experimental results show that our reverse pruning outperforms previous work with 29.01% in Top-1 accuracy on perturbed CIFAR-10. However, reverse pruning does not guarantee benign samples. To relax this problem, we further conducted experiments by adding a regularization term for the high magnitude weights. With adding the regularization term, we also applied conventional pruning to ensure the robustness of DNNs.

• 반도체디스플레이기술학회지
• /
• 제21권1호
• /
• pp.75-78
• /
• 2022

Deep Neural Networks (DNN) has become an essential data processing architecture for the implementation of multiple computer vision tasks. Recently, DNN-based algorithms achieve much higher recognition accuracy than traditional algorithms based on shallow learning. However, training and inference DNNs require huge computational capabilities than daily usage purposes of computers. Moreover, with increased size and depth of DNNs, CPUs may be unsatisfactory since they use serial processing by default. GPUs are the solution that come up with greater speed compared to CPUs because of their Parallel Processing/Computation nature. In this paper, we analyze the inference time complexity of DNNs using well-known computer vision library, OpenCV. We measure and analyze inference time complexity for three cases, CPU, GPU-Float32, and GPU-Float16.

• Current Optics and Photonics
• /
• 제8권1호
• /
• pp.80-85
• /
• 2024

Numerical values of the Auger coefficient and the free carrier absorption (FCA) coefficient are extracted by applying deep neural networks (DNNs) to the L-I characteristics of 850 nm GaAs/AlGaAs laser diodes. Two elemental DNNs are used to extract each coefficient sequentially. The fidelity of the extracted values is established through meticulous correlation of L-I characteristics bridging the realms of simulations and measurements. The methodology presented in this paper offers a way to accurately extract the Auger and FCA coefficients, which were traditionally treated as fitting parameters. It is anticipated that this approach will be applicable to other types of opto-electronic devices as well.

• 대한의용생체공학회:의공학회지
• /
• 제41권2호
• /
• pp.107-119
• /
• 2020

The deep neural networks (DNN) that can replicate the behavior of the human expert who recognizes the characteristics of ECG waveform have been developed and studied to analyze ECG. However, although the existing DNNs can not provide the explanations for their decisions, those trials have attempted to determine whether patients have certain diseases or not and those decisions could not be accepted because of the absence of relating theoretical basis. In addition, these DNNs required a lot of training data to obtain sufficient accuracy in spite of the difficulty in the acquisition of relating clinical data. In this study, a small-sized continuous data processing DNN (C-DNN) was suggested to determine the simple characteristics of ECG wave that were not required additional explanations about its decisions and the C-DNN can be easily trained with small training data. Although it can analyze small input data that was selected in narrow region on whole ECG, it can continuously scan all ECG data and find important points such as start and end points of P, QRS and T waves within a short time. The star and end points of ECG waves determined by the C-DNNs were compared with the results performed by human experts to estimate the accuracies of the C-DNNs. The C-DNN has 150 inputs, 51 outputs, two hidden layers and one output layer. To find the start and end points, two C-DNNs were trained through deep learning technology and applied to a parameter acquisition algorithms. 12 lead ECG data measured in four patients and obtained through PhysioNet was processed to make training data by human experts. The accuracy of the C-DNNs were evaluated with extra data that were not used at deep learning by comparing the results between C-DNNs and human experts. The averages of the time differences between the C-DNNs and experts were 0.1 msec and 13.5 msec respectively and those standard deviations were 17.6 msec and 15.7 msec. The final step combining the results of C-DNN through the waveforms of 12 leads was successfully determined all 33 waves without error that the time differences of human experts decision were over 20 msec. The reliable decision of the ECG wave's start and end points benefits the acquisition of accurate ECG parameters such as the wave lengths, amplitudes and intervals of P, QRS and T waves.

• 한국정보과학회 언어공학연구회:학술대회논문집(한글 및 한국어 정보처리)
• /
• 한국정보과학회언어공학연구회 2017년도 제29회 한글 및 한국어 정보처리 학술대회
• /
• pp.238-241
• /
• 2017

본 논문에서는 구글 학술 검색 기반의 데이터를 이용하여 질병과 폐질환과 관련된 바이오마커 단어의 유사도를 계산하는 방법을 제안한다. 질병과 바이오마커의 유사도를 계산할 때, 각 단어의 구글 학술 검색의 검색 결과를 이용하였다. 이를 통해 폐질환 관련 바이오마커와 다른 질병간의 관계를 파악하고자 하며, 의료 전문가에게 폐질환 관련 바이오마커와 다른 질병간의 새로운 관계를 제시하고자 한다. 이러한 데이터를 이용하여 계산한 결과, Wor2Vec의 결과를 이용한 코사인 유사도의 결과와 상관 계수가 약 0.64로 상당히 높은 상관 관계를 확인할 수 있었다. 따라서 이 방법을 통해 질병과 바이오마커의 관계를 파악하고자 하였다. 또한 Word2Vec을 이용한 질병과 바이오마커 단어의 벡터 값과 단어 유사도 계산 방법의 결과를 이용한 Deep Neural Networks (DNNs) 모델을 구축하고자 하며, 이를 통해 자동적으로 유사도를 분석하고자 하였다.

• 한국어정보학회:학술대회논문집
• /
• 한국어정보학회 2017년도 제29회 한글및한국어정보처리학술대회
• /
• pp.238-241
• /
• 2017

본 논문에서는 구글 학술 검색 기반의 데이터를 이용하여 질병과 폐질환과 관련된 바이오마커 단어의 유사도를 계산하는 방법을 제안한다. 질병과 바이오마커의 유사도를 계산할 때, 각 단어의 구글 학술 검색의 검색 결과를 이용하였다. 이를 통해 폐질환 관련 바이오마커와 다른 질병간의 관계를 파악하고자 히며, 의료 전문가에게 폐질환 관련 바이오마커와 다른 질병간의 새로운 관계를 제시하고자 한다. 이러한 데이터를 이용하여 계산한 결과, Wor2Vec의 결과를 이용한 코사인 유사도의 결과와 상관 계수가 약 0.64로 상당히 높은 상관 관계를 확인할 수 있었다. 따라서 이 방법을 통해 질병과 바이오마커의 관계를 파악하고자 하였다. 또한 Word2Vec을 이용한 질병과 바이오마커 단어의 벡터 값과 단어 유사도 계산 방법의 결과를 이용한 Deep Neural Networks (DNNs) 모델을 구축하고자 하며, 이를 통해 자동적으로 유사도를 분석하고자 하였다.

• Smart Structures and Systems
• /
• 제29권1호
• /
• pp.141-151
• /
• 2022

In recent years, the industry and research communities have focused on developing autonomous crack inspection approaches, which mainly include image acquisition and crack detection. In these approaches, mobile devices such as cameras, drones or smartphones are utilized as sensing platforms to acquire structural images, and the deep learning (DL)-based methods are being developed as important crack detection approaches. However, the process of image acquisition and collection is time-consuming, which delays the inspection. Also, the present mobile devices such as smartphones can be not only a sensing platform but also a computing platform that can be embedded with deep neural networks (DNNs) to conduct on-site crack detection. Due to the limited computing resources of mobile devices, the size of the DNNs should be reduced to improve the computational efficiency. In this study, an architecture called pruned crack recognition network (PCR-Net) was developed for the detection of structural cracks. A dataset containing 11000 images was established based on the raw images from bridge inspections. A pruning method was introduced to reduce the size of the base architecture for the optimization of the model size. Comparative studies were conducted with image processing techniques (IPTs) and other DNNs for the evaluation of the performance of the proposed PCR-Net. Furthermore, a modularly designed framework that integrated the PCR-Net was developed to realize a DL-based crack detection application for smartphones. Finally, on-site crack detection experiments were carried out to validate the performance of the developed system of smartphone-based detection of structural cracks.