• Title/Summary/Keyword: Entropy model

Search Result 485, Processing Time 0.024 seconds

A Complexity Metric for Web Documentation Based on Entropy (엔트로피를 기반으로한 Web 문서들의 복잡도 척도)

  • Kim, Kap-Su
    • Journal of The Korean Association of Information Education
    • /
    • v.2 no.2
    • /
    • pp.260-268
    • /
    • 1998
  • In this paper, I propose a metric model for measuring complexity of Web documentations which are wrote by HTML and XML. The complexity of Web documentation has effect on documentation understandability which is an important metric in maintenance and reusing of Web documentation. The understandable documents have more effect on WEI. The proposed metric uses the entropy to represent the degree of information flows between Web documentations. The proposed documentation complexity measures the information flows in a Web document based on the information passing relationship between Web document files. I evaluate the proposed metric by using the complexity properties proposed by Weyuker, and measure the document complexity. I show effectiveness of analyzing the correlation between the number of document file and document complexity.

  • PDF

Influence on overfitting and reliability due to change in training data

  • Kim, Sung-Hyeock;Oh, Sang-Jin;Yoon, Geun-Young;Jung, Yong-Gyu;Kang, Min-Soo
    • International Journal of Advanced Culture Technology
    • /
    • v.5 no.2
    • /
    • pp.82-89
    • /
    • 2017
  • The range of problems that can be handled by the activation of big data and the development of hardware has been rapidly expanded and machine learning such as deep learning has become a very versatile technology. In this paper, mnist data set is used as experimental data, and the Cross Entropy function is used as a loss model for evaluating the efficiency of machine learning, and the value of the loss function in the steepest descent method is We applied the GradientDescentOptimize algorithm to minimize and updated weight and bias via backpropagation. In this way we analyze optimal reliability value corresponding to the number of exercises and optimal reliability value without overfitting. And comparing the overfitting time according to the number of data changes based on the number of training times, when the training frequency was 1110 times, we obtained the result of 92%, which is the optimal reliability value without overfitting.

Chinese Prosody Generation Based on C-ToBI Representation for Text-to-Speech (음성합성을 위한 C-ToBI기반의 중국어 운율 경계와 F0 contour 생성)

  • Kim, Seung-Won;Zheng, Yu;Lee, Gary-Geunbae;Kim, Byeong-Chang
    • MALSORI
    • /
    • no.53
    • /
    • pp.75-92
    • /
    • 2005
  • Prosody Generation Based on C-ToBI Representation for Text-to-SpeechSeungwon Kim, Yu Zheng, Gary Geunbae Lee, Byeongchang KimProsody modeling is critical in developing text-to-speech (TTS) systems where speech synthesis is used to automatically generate natural speech. In this paper, we present a prosody generation architecture based on Chinese Tone and Break Index (C-ToBI) representation. ToBI is a multi-tier representation system based on linguistic knowledge to transcribe events in an utterance. The TTS system which adopts ToBI as an intermediate representation is known to exhibit higher flexibility, modularity and domain/task portability compared with the direct prosody generation TTS systems. However, the cost of corpus preparation is very expensive for practical-level performance because the ToBI labeled corpus has been manually constructed by many prosody experts and normally requires a large amount of data for accurate statistical prosody modeling. This paper proposes a new method which transcribes the C-ToBI labels automatically in Chinese speech. We model Chinese prosody generation as a classification problem and apply conditional Maximum Entropy (ME) classification to this problem. We empirically verify the usefulness of various natural language and phonology features to make well-integrated features for ME framework.

  • PDF

Comparison of Different CNN Models in Tuberculosis Detecting

  • Liu, Jian;Huang, Yidi
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.14 no.8
    • /
    • pp.3519-3533
    • /
    • 2020
  • Tuberculosis is a chronic and delayed infection which is easily experienced by young people. According to the statistics of the World Health Organization (WHO), there are nearly ten million fell ill with tuberculosis and a total of 1.5 million people died from tuberculosis in 2018 (including 251000 people with HIV). Tuberculosis is the largest single infectious pathogen that leads to death. In order to help doctors with tuberculosis diagnosis, we compare the tuberculosis classification abilities of six popular convolutional neural network (CNN) models in the same data set to find the best model. Before training, we optimize three parts of CNN to achieve better results. We employ sigmoid function to replace the step function as the activation function. What's more, we use binary cross entropy function as the cost function to replace traditional quadratic cost function. Finally, we choose stochastic gradient descent (SGD) as gradient descent algorithm. From the results of our experiments, we find that Densenet121 is most suitable for tuberculosis diagnosis and achieve a highest accuracy of 0.835. The optimization and expansion depend on the increase of data set and the improvements of Densenet121.

A Framework for Semantic Interpretation of Noun Compounds Using Tratz Model and Binary Features

  • Zaeri, Ahmad;Nematbakhsh, Mohammad Ali
    • ETRI Journal
    • /
    • v.34 no.5
    • /
    • pp.743-752
    • /
    • 2012
  • Semantic interpretation of the relationship between noun compound (NC) elements has been a challenging issue due to the lack of contextual information, the unbounded number of combinations, and the absence of a universally accepted system for the categorization. The current models require a huge corpus of data to extract contextual information, which limits their usage in many situations. In this paper, a new semantic relations interpreter for NCs based on novel lightweight binary features is proposed. Some of the binary features used are novel. In addition, the interpreter uses a new feature selection method. By developing these new features and techniques, the proposed method removes the need for any huge corpuses. Implementing this method using a modular and plugin-based framework, and by training it using the largest and the most current fine-grained data set, shows that the accuracy is better than that of previously reported upon methods that utilize large corpuses. This improvement in accuracy and the provision of superior efficiency is achieved not only by improving the old features with such techniques as semantic scattering and sense collocation, but also by using various novel features and classifier max entropy. That the accuracy of the max entropy classifier is higher compared to that of other classifiers, such as a support vector machine, a Na$\ddot{i}$ve Bayes, and a decision tree, is also shown.

Learning Text Chunking Using Maximum Entropy Models (최대 엔트로피 모델을 이용한 텍스트 단위화 학습)

  • Park, Seong-Bae;Zhang, Byoung-Tak
    • Annual Conference on Human and Language Technology
    • /
    • 2001.10d
    • /
    • pp.130-137
    • /
    • 2001
  • 최대 엔트로피 모델(maximum entropy model)은 여러 가지 자연언어 문제를 학습하는데 성공적으로 적용되어 왔지만, 두 가지의 주요한 문제점을 가지고 있다. 그 첫번째 문제는 해당 언어에 대한 많은 사전 지식(prior knowledge)이 필요하다는 것이고, 두번째 문제는 계산량이 너무 많다는 것이다. 본 논문에서는 텍스트 단위화(text chunking)에 최대 엔트로피 모델을 적용하는 데 나타나는 이 문제점들을 해소하기 위해 새로운 방법을 제시한다. 사전 지식으로, 간단한 언어 모델로부터 쉽게 생성된 결정트리(decision tree)에서 자동적으로 만들어진 규칙을 사용한다. 따라서, 제시된 방법에서의 최대 엔트로피 모델은 결정트리를 보강하는 방법으로 간주될 수 있다. 계산론적 복잡도를 줄이기 위해서, 최대 엔트로피 모델을 학습할 때 일종의 능동 학습(active learning) 방법을 사용한다. 전체 학습 데이터가 아닌 일부분만을 사용함으로써 계산 비용은 크게 줄어 들 수 있다. 실험 결과, 제시된 방법으로 결정트리의 오류의 수가 반으로 줄었다. 대부분의 자연언어 데이터가 매우 불균형을 이루므로, 학습된 모델을 부스팅(boosting)으로 강화할 수 있다. 부스팅을 한 후 제시된 방법은 전문가에 의해 선택된 자질로 학습된 최대 엔트로피 모델보다 졸은 성능을 보이며 지금까지 보고된 기계 학습 알고리즘 중 가장 성능이 좋은 방법과 비슷한 성능을 보인다 텍스트 단위화가 일반적으로 전체 구문분석의 전 단계이고 이 단계에서의 오류가 다음 단계에서 복구될 수 없으므로 이 성능은 텍스트 단위화에서 매우 의미가 길다.

  • PDF

A New Robust Signal Recognition Approach Based on Holder Cloud Features under Varying SNR Environment

  • Li, Jingchao
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.9 no.12
    • /
    • pp.4934-4949
    • /
    • 2015
  • The unstable characteristic values of communication signals along with the varying SNR (Signal Noise Ratio) environment make it difficult to identify the modulations of signals. Most of relevant literature revolves around signal recognition under stable SNR, and not applicable for signal recognition at varying SNR. To solve the problem, this research developed a novel communication signal recognition algorithm based on Holder coefficient and cloud theory. In this algorithm, the two-dimensional (2D) Holder coefficient characteristics of communication signals were firstly calculated, and then according to the distribution characteristics of Holder coefficient under varying SNR environment, the digital characteristics of cloud model such as expectation, entropy, and hyper entropy are calculated to constitute the three-dimensional (3D) digital cloud characteristics of Holder coefficient value, which aims to improve the recognition rate of the communication signals. Compared with traditional algorithms, the developed algorithm can describe the signals' features more accurately under varying SNR environment. The results from the numerical simulation show that the developed 3D feature extraction algorithm based on Holder coefficient cloud features performs better anti-noise ability, and the classifier based on interval gray relation theory can achieve a recognition rate up to 84.0%, even when the SNR varies from -17dB to -12dB.

Adsorption of Non-degradable Eosin Y by Activated Carbon (활성탄에 의한 난분해성 염료인 Eosin Y의 흡착)

  • Lee, Min-Gyu;Kam, Sang-Kyu;Suh, Keun-Hak
    • Journal of Environmental Science International
    • /
    • v.21 no.5
    • /
    • pp.623-631
    • /
    • 2012
  • The adsorption behavior of Eosin Y on activated carbon (AC) in batch system was investigated. The adsorption isotherm could be well fitted by the Langmuir adsorption equation. The kinetics of adsorption followed the pseudo-second-order model. The temperature variation was used to evaluate the values of free energy (${\Delta}G^{\circ}$), enthalpy (${\Delta}H^{\circ}$) and entropy (${\Delta}S^{\circ}$). The positive value of enthalpy change ${\Delta}H^{\circ}$ for the process confirms the endothermic nature of the process and more favourable at higher temperature, the positive entropy of adsorption ${\Delta}S^{\circ}$ reflects the affinity of the AC material toward Eosin Y and the negative free energy values ${\Delta}G^{\circ}$ indicate that the adsorption process is spontaneous. With the increase of the amount of AC, removal efficiency of Eosin Y was increased, but adsorption capacity was decreased. And adsorption capacity was increased with the decrease of particle size. With the increase of the amount of AC, removal efficiency of Eosin Y was increased, but adsorption capacity was decreased. And adsorption capacity was increased with the decrease of particle size.

Interaction of Wool-Keratine Membrane with Methyl Orange and It's Homologs over the Temperature Range 60~9$0^{\circ}C$ (양모―케라틴 유도체막과 메틸오렌지 및 그 동족체와의 고온영역에서의 상호작용)

  • Jeon, Jae Hong;Lee, Hwa Sun;Kim, Gong Ju
    • Textile Coloration and Finishing
    • /
    • v.7 no.2
    • /
    • pp.40-46
    • /
    • 1995
  • In order to study the dyeability of wool S-cyano ethylated wool-keratine(SCEK) as a model compound of wool was prepared from the reaction of reduced merino wool fiber and acrylonitrile. The binding of acid dyes(methyl orange and it's homologs) by SCEK over the temperature 60~9$0^{\circ}C$ were investigated. The first binding constants and the thermodynamic parameters in the course of the binding were evaluated. It was found that at the 60~9$0^{\circ}C$ range complex formation between the dye and SCEK is associated with an exothermic enthalpy change and a positive entropy change. The enthalpy and entropy changes of the binding are of the order of -4.5 kcal/mole and 8.5 eu, respectively, for each dye measured. Thus the binding is mainly enthalpy-controlled. Furthermore the effect of the alkyl chain length of the dye on both the ΔH$^{\circ}$and ΔS$^{\circ}$value is not prounced. Also temperature dependences of the ΔH$^{\circ}$and ΔS$^{\circ}$values were not obserbed.

  • PDF

Techniques for improving performance of POS tagger based on Maximum Entropy Model (최대 엔트로피 모텔 기반 품사 태거의 성능 향상 기법)

  • Cho, Min-Hee;Kim, Myoung-Sun;Park, Jae-Han;Park, Eui-Kyu;Ra, Dong-Yul
    • Annual Conference on Human and Language Technology
    • /
    • 2004.10d
    • /
    • pp.73-81
    • /
    • 2004
  • 한국어에서의 품사 결정 문제는 형태론적 중의성 문제도 있지만, 영어에는 발생하지 않는 동품사 중의성 문제로 더 까다롭다. 이러한 문제들은 어휘 문맥을 고려하지 않고서는 해결하기 어렵다. 통계 자료 부족 문제에 쉽게 대처하는 모델이 필요하며 문맥에 따른 품사를 결정하고자 할 때 서로 다른 형태의 여러 가지 어휘 문맥 정보를 반영할 수 있는 모델이 필요하다. 본 논문에서는 이런 점에 가장 적합한 최대 엔트로피(maximum entropy : ME) 모델을 품사태깅 작업에 이용하는 문제에 대해 다룬다. 어휘 문맥 정보를 이용하기 위한 자질함수가 매우 많아지는 문제에 대처하기 위해 필요에 따라 어휘 문맥 정보를 사전화 한다. 본 시스템의 특징으로는 어절 단위 품사 태깅을 위한 처리 기법. 어절의 형태소 분석열에 대한 어절 내부 확률 계산. ME 모델의 정규화 과정 생략에 의한 성능 향상, 디코딩 경로의 확장과 같은 점들이 있다. 실험을 통하여 본 연구의 기법이 높은 성능의 시스템을 달성할 수 있음을 알게 되었다.

  • PDF