• Title/Summary/Keyword: Short text

Search Result 253, Processing Time 0.025 seconds

Research on Keyword-Overlap Similarity Algorithm Optimization in Short English Text Based on Lexical Chunk Theory

  • Na Li;Cheng Li;Honglie Zhang
    • Journal of Information Processing Systems
    • /
    • v.19 no.5
    • /
    • pp.631-640
    • /
    • 2023
  • Short-text similarity calculation is one of the hot issues in natural language processing research. The conventional keyword-overlap similarity algorithms merely consider the lexical item information and neglect the effect of the word order. And some of its optimized algorithms combine the word order, but the weights are hard to be determined. In the paper, viewing the keyword-overlap similarity algorithm, the short English text similarity algorithm based on lexical chunk theory (LC-SETSA) is proposed, which introduces the lexical chunk theory existing in cognitive psychology category into the short English text similarity calculation for the first time. The lexical chunks are applied to segment short English texts, and the segmentation results demonstrate the semantic connotation and the fixed word order of the lexical chunks, and then the overlap similarity of the lexical chunks is calculated accordingly. Finally, the comparative experiments are carried out, and the experimental results prove that the proposed algorithm of the paper is feasible, stable, and effective to a large extent.

Multi-layered attentional peephole convolutional LSTM for abstractive text summarization

  • Rahman, Md. Motiur;Siddiqui, Fazlul Hasan
    • ETRI Journal
    • /
    • v.43 no.2
    • /
    • pp.288-298
    • /
    • 2021
  • Abstractive text summarization is a process of making a summary of a given text by paraphrasing the facts of the text while keeping the meaning intact. The manmade summary generation process is laborious and time-consuming. We present here a summary generation model that is based on multilayered attentional peephole convolutional long short-term memory (MAPCoL; LSTM) in order to extract abstractive summaries of large text in an automated manner. We added the concept of attention in a peephole convolutional LSTM to improve the overall quality of a summary by giving weights to important parts of the source text during training. We evaluated the performance with regard to semantic coherence of our MAPCoL model over a popular dataset named CNN/Daily Mail, and found that MAPCoL outperformed other traditional LSTM-based models. We found improvements in the performance of MAPCoL in different internal settings when compared to state-of-the-art models of abstractive text summarization.

Combining Distributed Word Representation and Document Distance for Short Text Document Clustering

  • Kongwudhikunakorn, Supavit;Waiyamai, Kitsana
    • Journal of Information Processing Systems
    • /
    • v.16 no.2
    • /
    • pp.277-300
    • /
    • 2020
  • This paper presents a method for clustering short text documents, such as news headlines, social media statuses, or instant messages. Due to the characteristics of these documents, which are usually short and sparse, an appropriate technique is required to discover hidden knowledge. The objective of this paper is to identify the combination of document representation, document distance, and document clustering that yields the best clustering quality. Document representations are expanded by external knowledge sources represented by a Distributed Representation. To cluster documents, a K-means partitioning-based clustering technique is applied, where the similarities of documents are measured by word mover's distance. To validate the effectiveness of the proposed method, experiments were conducted to compare the clustering quality against several leading methods. The proposed method produced clusters of documents that resulted in higher precision, recall, F1-score, and adjusted Rand index for both real-world and standard data sets. Furthermore, manual inspection of the clustering results was conducted to observe the efficacy of the proposed method. The topics of each document cluster are undoubtedly reflected by members in the cluster.

Reading Don Lee's Yellow as a Short Story Cycle ("단편소설집의 사이클"로서 단 리의 『옐로우』 연구)

  • Lee, Su Mee
    • Journal of English Language & Literature
    • /
    • v.57 no.5
    • /
    • pp.727-755
    • /
    • 2011
  • In this paper, I'll try to read Don Lee's Yellow intertextually with a more canonical text, Sherwood Anderson's Winesburg, Ohio, in order to see what kind of traditions and techniques Yellow references and/or rewrites as a way of tracking this production. Yellow's formal properties as a short story cycle are established through its use of particular conventions. For instance, Yellow follows the short story cycle model that includes the assemblage of recurring characters into one locale. Yellow's characters are all connected to and at some point located in the fictional small town of Rosarita Bay, California. The text form aligns it with established literary conventions and traditions and suggests the author's reliance upon or trust in those modes. Yellow's setting in a small town alludes to and has often been compared to Anderson's Winesburg, Ohio, which is perhaps one of the most well-known and extensively discussed short story cycles in American literature. Also following convention is Lee's construction of Rosarita Bay and the text's third person narrator as a member of that town. Both Rosarita Bay and the narrator become important figures through the related-tale nature of the text. The method of story-telling is similar to how the town Winesburg and its "seemingly sympathetic and non-overtly judgmental" narrator are operational in Anderson's text. In sum, Yellow is opportune for intertextual reading largely because it is a collection of stories that create a linked series.

Short Text Classification for Job Placement Chatbot by T-EBOW (T-EBOW를 이용한 취업알선 챗봇용 단문 분류 연구)

  • Kim, Jeongrae;Kim, Han-joon;Jeong, Kyoung Hee
    • Journal of Internet Computing and Services
    • /
    • v.20 no.2
    • /
    • pp.93-100
    • /
    • 2019
  • Recently, in various business fields, companies are concentrating on providing chatbot services to various environments by adding artificial intelligence to existing messenger platforms. Organizations in the field of job placement also require chatbot services to improve the quality of employment counseling services and to solve the problem of agent management. A text-based general chatbot classifies input user sentences into learned sentences and provides appropriate answers to users. Recently, user sentences inputted to chatbots are inputted as short texts due to the activation of social network services. Therefore, performance improvement of short text classification can contribute to improvement of chatbot service performance. In this paper, we propose T-EBOW (Translation-Extended Bag Of Words), which is a method to add translation information as well as concept information of existing researches in order to strengthen the short text classification for employment chatbot. The performance evaluation results of the T-EBOW applied to the machine learning classification model are superior to those of the conventional method.

Using Highly Secure Data Encryption Method for Text File Cryptography

  • Abu-Faraj, Mua'ad M.;Alqadi, Ziad A.
    • International Journal of Computer Science & Network Security
    • /
    • v.21 no.12
    • /
    • pp.53-60
    • /
    • 2021
  • Many standard methods are used for secret text files and secrete short messages cryptography, these methods are efficient when the text to be encrypted is small, and the efficiency will rapidly decrease when increasing the text size, also these methods sometimes have a low level of security, this level will depend on the PK length and sometimes it may be hacked. In this paper, a new method will be introduced to improve the data protection level by using a changeable secrete speech file to generate PK. Highly Secure Data Encryption (HSDE) method will be implemented and tested for data quality levels to ensure that the HSDE destroys the data in the encryption phase, and recover the original data in the decryption phase. Some standard methods of data cryptography will be implemented; comparisons will be done to justify the enhancements provided by the proposed method.

A Method for Short Text Classification using SNS Feature Information based on Markov Logic Networks (SNS 특징정보를 활용한 마르코프 논리 네트워크 기반의 단문 텍스트 분류 방법)

  • Lee, Eunji;Kim, Pankoo
    • Journal of Korea Multimedia Society
    • /
    • v.20 no.7
    • /
    • pp.1065-1072
    • /
    • 2017
  • As smart devices and social network services (SNSs) become increasingly pervasive, individuals produce large amounts of data in real time. Accordingly, studies on unstructured data analysis are actively being conducted to solve the resultant problem of information overload and to facilitate effective data processing. Many such studies are conducted for filtering inappropriate information. In this paper, a feature-weighting method considering SNS-message features is proposed for the classification of short text messages generated on SNSs, using Markov logic networks for category inference. The performance of the proposed method is verified through a comparison with an existing frequency-based classification methods.

Automatic proficiency assessment of Korean speech read aloud by non-natives using bidirectional LSTM-based speech recognition

  • Oh, Yoo Rhee;Park, Kiyoung;Jeon, Hyung-Bae;Park, Jeon Gue
    • ETRI Journal
    • /
    • v.42 no.5
    • /
    • pp.761-772
    • /
    • 2020
  • This paper presents an automatic proficiency assessment method for a non-native Korean read utterance using bidirectional long short-term memory (BLSTM)-based acoustic models (AMs) and speech data augmentation techniques. Specifically, the proposed method considers two scenarios, with and without prompted text. The proposed method with the prompted text performs (a) a speech feature extraction step, (b) a forced-alignment step using a native AM and non-native AM, and (c) a linear regression-based proficiency scoring step for the five proficiency scores. Meanwhile, the proposed method without the prompted text additionally performs Korean speech recognition and a subword un-segmentation for the missing text. The experimental results indicate that the proposed method with prompted text improves the performance for all scores when compared to a method employing conventional AMs. In addition, the proposed method without the prompted text has a fluency score performance comparable to that of the method with prompted text.

A study on the aesthetic elements of Chinese translated Korean novel - Focused on the mode of narrations in "An old well" written by Jeong Heui Oh (우리말 소설의 중국어 번역에서 미적요소의 재현문제(2) - '화법'에서 본 오정희의 『옛 우물』(『老井』))

  • Choi, Eun Jeong
    • Cross-Cultural Studies
    • /
    • v.26
    • /
    • pp.201-226
    • /
    • 2012
  • This essay exams the issues of aesthetic elements that come up when Korean novels get translated into Chinese language. The short story collection titled "An old well" written by Jeong Heui Oh in both languages are compared and analyzed by focusing on the mode of narrations. There are various narrative modes in "An old well". Each narrative mode properly functions for aesthetic effects and drawing meanings. In short, we can find a way to grasp its leitmotif the writer wants to indicate only when we carefully interpret the narrative modes in the original text. However, the narrative modes in Korean text have been simplified by changing its modes into direct narrative in Chinese-translated text. Thus the aesthetic effects in the original text have been spoiled and the Chinese text fails to deliver its meaning involved in the original narrative mode. Translation of novel invites consideration on both of its form and content on account of the text's uniqueness. Accordingly, a close examination and study of the original text should be completed beforehand.

Use of Word Clustering to Improve Emotion Recognition from Short Text

  • Yuan, Shuai;Huang, Huan;Wu, Linjing
    • Journal of Computing Science and Engineering
    • /
    • v.10 no.4
    • /
    • pp.103-110
    • /
    • 2016
  • Emotion recognition is an important component of affective computing, and is significant in the implementation of natural and friendly human-computer interaction. An effective approach to recognizing emotion from text is based on a machine learning technique, which deals with emotion recognition as a classification problem. However, in emotion recognition, the texts involved are usually very short, leaving a very large, sparse feature space, which decreases the performance of emotion classification. This paper proposes to resolve the problem of feature sparseness, and largely improve the emotion recognition performance from short texts by doing the following: representing short texts with word cluster features, offering a novel word clustering algorithm, and using a new feature weighting scheme. Emotion classification experiments were performed with different features and weighting schemes on a publicly available dataset. The experimental results suggest that the word cluster features and the proposed weighting scheme can partly resolve problems with feature sparseness and emotion recognition performance.