• Title/Summary/Keyword: Text data

Search Result 2,953, Processing Time 0.032 seconds

A Two-Phase On-Device Analysis for Gender Prediction of Mobile Users Using Discriminative and Popular Wordsets (모바일 사용자의 성별 예측을 위한 식별 및 인기 단어 집합 기반 2단계 기기 내 분석)

  • Choi, Yerim;Park, Kyuyon;Kim, Solee;Park, Jonghun
    • The Journal of Society for e-Business Studies
    • /
    • v.21 no.1
    • /
    • pp.65-77
    • /
    • 2016
  • As respecting one's privacy becomes an important issue in mobile device data analysis, on-device analysis is getting attention, in which the data analysis is conducted inside a mobile device without sending data from the device to outside. One possible application of the on-device analysis is gender prediction using text data in mobile devices, such as text messages, search keyword, website bookmarks, and contact, which are highly private, and the limited computing power of mobile devices can be addressed by utilizing the word comparison method, where words are selected beforehand and delivered to a mobile device of a user to determine the user's gender by matching mobile text data and the selected words. Moreover, it is known that performing prediction after filtering instances using definite evidences increases accuracy and reduces computational complexity. In this regard, we propose a two-phase approach to on-device gender prediction, where both discriminability and popularity of a word are sequentially considered. The proposed method performs predictions using a few highly discriminative words for all instances and popular words for unclassified instances from the previous prediction. From the experiments conducted on real-world dataset, the proposed method outperformed the compared methods.

Comparison of responses to issues in SNS and Traditional Media using Text Mining -Focusing on the Termination of Korea-Japan General Security of Military Information Agreement(GSOMIA)- (텍스트 마이닝을 이용한 SNS와 언론의 이슈에 대한 반응 비교 -"한일군사정보보호협정(GSOMIA) 종료"를 중심으로-)

  • Lee, Su Ryeon;Choi, Eun Jung
    • Journal of Digital Convergence
    • /
    • v.18 no.2
    • /
    • pp.277-284
    • /
    • 2020
  • Text mining is a representative method of big data analysis that extracts meaningful information from unstructured and large amounts of text data. Social media such as Twitter generates hundreds of thousands of data per second and acts as a one-person media that instantly and directly expresses public opinions and ideas. The traditional media are delivering informations, criticizing society, and forming public opinions. For this, we compare the responses of SNS with the responses of media on the issue of the termination of the Korea-Japan GSOMIA (General Security of Military Information Agreement), one of the domestic issues in the second half of 2019. Data collected from 201,728 tweets and 20,698 newspaper articles were analyzed by sentiment analysis, association keyword analysis, and cluster analysis. As a result, SNS tends to respond positively to this issue, and the media tends to react negatively. In association keyword analysis, SNS shows positive views on domestic issues such as "destruction, decision, we," while the media shows negative views on external issues such as "disappointment, regret, concern". SNS is faster and more powerful than media when studying or creating social trends and opinions, rather than the function of information delivery. This can complement the role of the media that reflects public perception.

Similar Contents Recommendation Model Based On Contents Meta Data Using Language Model (언어모델을 활용한 콘텐츠 메타 데이터 기반 유사 콘텐츠 추천 모델)

  • Donghwan Kim
    • Journal of Intelligence and Information Systems
    • /
    • v.29 no.1
    • /
    • pp.27-40
    • /
    • 2023
  • With the increase in the spread of smart devices and the impact of COVID-19, the consumption of media contents through smart devices has significantly increased. Along with this trend, the amount of media contents viewed through OTT platforms is increasing, that makes contents recommendations on these platforms more important. Previous contents-based recommendation researches have mostly utilized metadata that describes the characteristics of the contents, with a shortage of researches that utilize the contents' own descriptive metadata. In this paper, various text data including titles and synopses that describe the contents were used to recommend similar contents. KLUE-RoBERTa-large, a Korean language model with excellent performance, was used to train the model on the text data. A dataset of over 20,000 contents metadata including titles, synopses, composite genres, directors, actors, and hash tags information was used as training data. To enter the various text features into the language model, the features were concatenated using special tokens that indicate each feature. The test set was designed to promote the relative and objective nature of the model's similarity classification ability by using the three contents comparison method and applying multiple inspections to label the test set. Genres classification and hash tag classification prediction tasks were used to fine-tune the embeddings for the contents meta text data. As a result, the hash tag classification model showed an accuracy of over 90% based on the similarity test set, which was more than 9% better than the baseline language model. Through hash tag classification training, it was found that the language model's ability to classify similar contents was improved, which demonstrated the value of using a language model for the contents-based filtering.

Proposal of Brand Evaluation Map through Big Data : Focus on The Hyundai Motor's Product Evaluation (빅데이터를 통한 브랜드 평가 맵 제안 : 현대자동차 제품 평가 중심으로)

  • Youn, Dae Myung;Lee, Yong Hyuck;Lee, Bong Gyou
    • Journal of Information Technology Services
    • /
    • v.19 no.4
    • /
    • pp.1-11
    • /
    • 2020
  • Through text mining, sentiment analysis, and semiotics analysis, this study aims to reinterpret the meaning of user emotional words and related words to derive strategic elements of brand and design. After selecting a local car manufacturer whose user opinion on the brand is a clear topic, web-crawl the car comments of the manufacturer directly created by the users online. Then, analyze the extracted morphology and its associated words and convert them to fit the marketing mix theory. Through this process, propose a methodology that allows consumers to supplement and improve brand elements with negative sensibilities, and to inherit elements with positive sensibilities and manage brands reasonably. In particular, the Map presented in this study are considered to be fully utilized as information for overall brand management.

WWW Based Instruction Systems for English Learning: GAIA

  • Park, Phan-Woo
    • Journal of The Korean Association of Information Education
    • /
    • v.3 no.2
    • /
    • pp.113-119
    • /
    • 2000
  • I studied a distance education model for English learning on the Internet. Basic WWW files, that contain courseware, are constructed with HTML, and functions, which are required in learning, are implemented with Java. Students and educators can access the preferred unit composed of the appropriate text, voice and image data by using a WWW browser at any time. The education system supports the automatic generation facility of English problems to practice reading and writing by making good use of the courseware data or various English text resources located on the Internet. Our system has functions to manage and control the flow of distance learning and to offer interaction between students and the system in a distributed environment. Educators can manage students' learning and can immediately be aware of who is attending and who is quitting the lesson in virtual space. Also, students and educators in different places can communicate and discuss a topic through the server. I implemented these functions, which are required in a client/server environment of distance education, with the use of Java. The URL for this system is "http://park.taegu-e.ac.kr" in the name of GAIA.

  • PDF

Analysis of the Online Review Based on the Theme Using the Hierarchical Attention Network (Hierarchical Attention Network를 활용한 주제에 따른 온라인 고객 리뷰 분석 모델)

  • Jang, In Ho;Park, Ki Yeon;Lee, Zoon Ky
    • Journal of Information Technology Services
    • /
    • v.17 no.2
    • /
    • pp.165-177
    • /
    • 2018
  • Recently, online commerces are becoming more common due to factors such as mobile technology development and smart device dissemination, and online review has a big influence on potential buyer's purchase decision. This study presents a set of analytical methodologies for understanding the meaning of customer reviews of products in online transaction. Using techniques currently developed in deep learning are implemented Hierarchical Attention Network for analyze meaning in online reviews. By using these techniques, we could solve time consuming pre-data analysis time problem and multiple topic problems. To this end, this study analyzes customer reviews of laptops sold in domestic online shopping malls. Our result successfully demonstrates over 90% classification accuracy. Therefore, this study classified the unstructured text data in the semantic analysis and confirmed the practical application possibility of the review analysis process.

Performance comparison of Text-Independent Speaker Recognizer Using VQ and GMM (VQ와 GMM을 이용한 문맥독립 화자인식기의 성능 비교)

  • Kim, Seong-Jong;Chung, Hoon;Chung, Ik-Joo
    • Speech Sciences
    • /
    • v.7 no.2
    • /
    • pp.235-244
    • /
    • 2000
  • This paper was focused on realizing the text-independent speaker recognizer using the VQ and GMM algorithm and studying the characteristics of the speaker recognizers that adopt these two algorithms. Because it was difficult ascertain the effect two algorithms have on the speaker recognizer theoretically, we performed the recognition experiments using various parameters and, as the result of the experiments, we could show that GMM algorithm had better recognition performance than VQ algorithm as following. The GMM showed better performance with small training data, and it also showed just a little difference of recognition rate as the kind of feature vectors and the length of input data vary. The GMM showed good recognition performance than the VQ on the whole.

  • PDF

An Experimental Study on Automatic Indexing for Hangeul Text (한글문헌의 자동색인에 관한 실험적 연구)

  • Ahn, Heyon-Soo
    • Journal of the Korean Society for information Management
    • /
    • v.3 no.2
    • /
    • pp.109-128
    • /
    • 1986
  • The explosive amount of information and various demands for it have led to the development of automatic indexing. Specially, in the HANGEUL data processing, the necessity of automatic indexing has been steadily increased. It is hypothesized that in the HANGEUL text, CHE-ON's only become key words and the CHE-ON is followed by JOSA. Through the morphological analysis the key words were selected from the titles and abstracts in the experimental data which consisted of 20 papers in "Journal of the Korea Society for Information Science."

  • PDF

Development of e-Mail Classifiers for e-Mail Response Management Systems (전자메일 자동관리 시스템을 위한 전자메일 분류기의 개발)

  • Kim, Kuk-Pyo;Kwon, Young-S.
    • Journal of Information Technology Services
    • /
    • v.2 no.2
    • /
    • pp.87-95
    • /
    • 2003
  • With the increasing proliferation of World Wide Web, electronic mail systems have become very widely used communication tools. Researches on e-mail classification have been very important in that e-mail classification system is a major engine for e-mail response management systems which mine unstructured e-mail messages and automatically categorize them. in this research we develop e-mail classifiers for e-mail Response Management Systems (ERMS) using naive bayesian learning and centroid-based classification. We analyze which method performs better under which conditions, comparing classification accuracies which may depend on the structure, the size of training data set and number of classes, using the different data set of an on-line shopping mall and a credit card company. The developed e-mail classifiers have been successfully implemented in practice. The experimental results show that naive bayesian learning performs better, while centroid-based classification is more robust in terms of classification accuracy.

The Use of MSVM and HMM for Sentence Alignment

  • Fattah, Mohamed Abdel
    • Journal of Information Processing Systems
    • /
    • v.8 no.2
    • /
    • pp.301-314
    • /
    • 2012
  • In this paper, two new approaches to align English-Arabic sentences in bilingual parallel corpora based on the Multi-Class Support Vector Machine (MSVM) and the Hidden Markov Model (HMM) classifiers are presented. A feature vector is extracted from the text pair that is under consideration. This vector contains text features such as length, punctuation score, and cognate score values. A set of manually prepared training data was assigned to train the Multi-Class Support Vector Machine and Hidden Markov Model. Another set of data was used for testing. The results of the MSVM and HMM outperform the results of the length based approach. Moreover these new approaches are valid for any language pairs and are quite flexible since the feature vector may contain less, more, or different features, such as a lexical matching feature and Hanzi characters in Japanese-Chinese texts, than the ones used in the current research.