• 한국언어정보학회:학술대회논문집
• /
• 한국언어정보학회 2002년도 Language, Information, and Computation Proceedings of The 16th Pacific Asia Conference
• /
• pp.28-35
• /
• 2002

Automatic identification of Chinese personal names in unrestricted texts is a key task in Chinese word segmentation, and can affect other NLP tasks such as word segmentation and information retrieval, if it is not properly addressed. This paper (1) demonstrates the problems of Chinese personal name identification in some If applications, (2) analyzes the structure of Chinese personal names, and (3) further presents the relevant processing strategies. The geographical differences of Chinese personal names between Beijing and Hong Kong are highlighted at the end. It shows that variation in names across different Chinese communities constitutes a critical factor in designing Chinese personal name Identification algorithm.

• Journal of Contemporary Eastern Asia
• /
• 제16권2호
• /
• pp.110-144
• /
• 2017

Unlike most languages, written Chinese has no spaces between words. Word segmentation must be performed before semantic network analysis can be conducted. This paper describes how to perform Chinese word segmentation using the Stanford Natural Language Processing group's Stanford Word Segmenter v. 3.8.0, released in June 2017.

• 한국언어정보학회:학술대회논문집
• /
• 한국언어정보학회 1995년도 Language, Information and Computation = Proceedings of the 10th Pacific Asia Conference, Hong Kong
• /
• pp.121-126
• /
• 1995

A new method for Chinese word segmentation named Conditional F'||'&'||'BMM (Forward and Backward Maximal Matching) which incorporates both bigram statistics (ie., mutual infonllation and difference of t-test between Chinese characters) and linguistic rules for ambiguity resolution is proposed in this paper The key characteristics of this model are the use of: (i) statistics which can be automatically derived from any raw corpus, (ii) a rule base for disambiguation with consistency and controlled size to be built up in a systematic way.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제15권6호
• /
• pp.2012-2030
• /
• 2021

Named Entity Recognition (NER) is used to identify entity nouns in the corpus such as Location, Person and Organization, etc. NER is also an important basic of research in various natural language fields. The processing of Chinese NER has some unique difficulties, for example, there is no obvious segmentation boundary between each Chinese character in a Chinese sentence. The Chinese NER task is often combined with Chinese word segmentation, and so on. In response to these problems, we summarize the recognition methods of Chinese NER. In this review, we first introduce the sequence labeling system and evaluation metrics of NER. Then, we divide Chinese NER methods into rule-based methods, statistics-based machine learning methods and deep learning-based methods. Subsequently, we analyze in detail the model framework based on deep learning and the typical Chinese NER methods. Finally, we put forward the current challenges and future research directions of Chinese NER technology.

• 한국언어정보학회:학술대회논문집
• /
• 한국언어정보학회 1998년도 Language, Information and Computation = Selected Papers from the 12th Pacific Asia Conference on Language, Information and Computation, Singapore
• /
• pp.212-217
• /
• 1998

• 한국언어정보학회:학술대회논문집
• /
• 한국언어정보학회 1998년도 Language, Information and Computation = Selected Papers from the 12th Pacific Asia Conference on Language, Information and Computation, Singapore
• /
• pp.218-229
• /
• 1998

• Industrial Engineering and Management Systems
• /
• 제15권3호
• /
• pp.224-230
• /
• 2016

The use of new words, numerous spoken languages, and abbreviations on the Internet is extensive. As such, automatically acquiring words for the purpose of analyzing Internet content is very difficult. In a previous study, we proposed a method for Japanese word segmentation using character N-grams. The previously proposed method is based on a simple state-transition model that is established under the assumption that the input document is described based on four states (denoted as A, B, C, and D) specified beforehand: state A represents words (nouns, verbs, etc.); state B represents statement separators (punctuation marks, conjunctions, etc.); state C represents postpositions (namely, words that follow nouns); and state D represents prepositions (namely, words that precede nouns). According to this state-transition model, based on the states applied to each pseudo-word, we search the document from beginning to end for an accessible pattern. In other words, the process of this transition detects some words during the search. In the present paper, we perform experiments based on the proposed word acquisition algorithm using Japanese and Chinese newspaper articles. These articles were obtained from Japan's Kyoto University and the Chinese People's Daily. The proposed method does not depend on the language structure. If text documents are expressed in Unicode the proposed method can, using the same algorithm, obtain words in Japanese and Chinese, which do not contain spaces between words. Hence, we demonstrate that the proposed method is language independent.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제16권6호
• /
• pp.1833-1848
• /
• 2022

Named entity recognition (NER) is an important basic task in the field of Natural Language Processing (NLP). Recently deep learning approaches by extracting word segmentation or character features have been proved to be effective for Chinese Named Entity Recognition (CNER). However, since this method of extracting features only focuses on extracting some of the features, it lacks textual information mining from multiple perspectives and dimensions, resulting in the model not being able to fully capture semantic features. To tackle this problem, we propose a novel Multi-view Semantic Feature Fusion Model (MSFM). The proposed model mainly consists of two core components, that is, Multi-view Semantic Feature Fusion Embedding Module (MFEM) and Multi-head Self-Attention Mechanism Module (MSAM). Specifically, the MFEM extracts character features, word boundary features, radical features, and pinyin features of Chinese characters. The acquired font shape, font sound, and font meaning features are fused to enhance the semantic information of Chinese characters with different granularities. Moreover, the MSAM is used to capture the dependencies between characters in a multi-dimensional subspace to better understand the semantic features of the context. Extensive experimental results on four benchmark datasets show that our method improves the overall performance of the CNER model.

• 한국의류산업학회지
• /
• 제18권3호
• /
• pp.317-326
• /
• 2016

The main purposes of this study were to segment Chinese consumers who purchase fashion products through internet commerce according to internet shopping values, to compare their online word-of-mouth acceptance and dissemination behavior, and to examine the demographic characteristics and purchase behavior of the segments. 715 questionnaires were collected through internet survey from January $19^{th}$ to March $16^{th}$, 2015 and a total of 488 were used for the final data analysis. The respondents were twenty to thirty nine years old men and women living in all over China. Hedonic and utilitarian shopping values were identified through factor analysis and based on the shopping values, the respondents were categorized into four groups-ambivalent shopping value group, hedonic shopping value group, utilitarian shopping value group and indifferent group. Among these groups, there were significant differences in terms of online word-of-mouth acceptance as well as dissemination level and motivation. In overall, ambivalent shopping value group showed high online word-of-mouth acceptance as well as dissemination motivation. The groups also showed significant differences in clothing selection criteria, frequently purchased internet shopping sites, online clothing shopping frequency and information sources. The groups also differed in terms of age, residential area, education level, occupation and income. However, there were no significant differences in gender and marital status among the groups.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제15권11호
• /
• pp.3991-4010
• /
• 2021

The Question Similarity Measurement of Chinese Crop Diseases and Insect Pests (QSM-CCD&IP) aims to judge the user's tendency to ask questions regarding input problems. The measurement is the basis of the Agricultural Knowledge Question and Answering (Q & A) system, information retrieval, and other tasks. However, the corpus and measurement methods available in this field have some deficiencies. In addition, error propagation may occur when the word boundary features and local context information are ignored when the general method embeds sentences. Hence, these factors make the task challenging. To solve the above problems and tackle the Question Similarity Measurement task in this work, a corpus on Chinese crop diseases and insect pests(CCDIP), which contains 13 categories, was established. Then, taking the CCDIP as the research object, this study proposes a Chinese agricultural text similarity matching model, namely, the AgrCQS. This model is based on mixed information extraction. Specifically, the hybrid embedding layer can enrich character information and improve the recognition ability of the model on the word boundary. The multi-scale local information can be extracted by multi-core convolutional neural network based on multi-weight (MM-CNN). The self-attention mechanism can enhance the fusion ability of the model on global information. In this research, the performance of the AgrCQS on the CCDIP is verified, and three benchmark datasets, namely, AFQMC, LCQMC, and BQ, are used. The accuracy rates are 93.92%, 74.42%, 86.35%, and 83.05%, respectively, which are higher than that of baseline systems without using any external knowledge. Additionally, the proposed method module can be extracted separately and applied to other models, thus providing reference for related research.