• Title/Summary/Keyword: Parsing Algorithm

Search Result 70, Processing Time 0.021 seconds

Fake News Checking Tool Based on Siamese Neural Networks and NLP (NLP와 Siamese Neural Networks를 이용한 뉴스 사실 확인 인공지능 연구)

  • Vadim, Saprunov;Kang, Sung-Won;Rhee, Kyung-hyune
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2022.05a
    • /
    • pp.627-630
    • /
    • 2022
  • Over the past few years, fake news has become one of the most significant problems. Since it is impossible to prevent people from spreading misinformation, people should analyze the news themselves. However, this process takes some time and effort, so the routine part of this analysis should be automated. There are many different approaches to this problem, but they only analyze the text and messages, ignoring the images. The fake news problem should be solved using a complex analysis tool to reach better performance. In this paper, we propose the approach of training an Artificial Intelligence using an unsupervised learning algorithm, combined with online data parsing tools, providing independence from subjective data set. Therefore it will be more difficult to spread fake news since people could quickly check if the news or article is trustworthy.

Improved Method for Learning Context-Free Grammar using Tabular representation

  • Jung, Soon-Ho
    • Journal of the Korea Society of Computer and Information
    • /
    • v.27 no.2
    • /
    • pp.43-51
    • /
    • 2022
  • In this paper, we suggest the method to improve the existing method leaning context-free grammar(CFG) using tabular representation(TBL) as a chromosome of genetic algorithm in grammatical inference and show the more efficient experimental result. We have two improvements. The first is to improve the formula to reflect the learning evaluation of positive and negative examples at the same time for the fitness function. The second is to classify partitions corresponding to TBLs generated from positive learning examples according to the size of the learning string, proceed with the evolution process by class, and adjust the composition ratio according to the success rate to apply the learning method linked to survival in the next generation. These improvements provide better efficiency than the existing method by solving the complexity and difficulty in the crossover and generalization steps between several individuals according to the size of the learning examples. We experiment with the languages proposed in the existing method, and the results show a rather fast generation rate that takes fewer generations to complete learning with the same success rate than the existing method. In the future, this method can be tried for extended CYK, and furthermore, it suggests the possibility of being applied to more complex parsing tables.

The signal processing algorithm of the Missile Flight Test Launch Control System (비행시험 발사통제 시스템의 신호처리 알고리즘)

  • Oh, Jino
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.19 no.8
    • /
    • pp.1965-1972
    • /
    • 2015
  • The Missile Flight Test Launch Control System is to operate in conjunction with the Fire Control System during flight test to guided weapons. Also, this is a system for the test control and situation monitoring depending on the type of guided weapons and testing purposes. Message structure, communication protocols, such as data types for interworking with the fire control system and the Missile Flight Test Launch Control System are defined in the Launch Control ICD(Interface Control Document). ICD are composed differently of each guided weapons system and each test object. Previously, in order to interwork with the Fire Control System, the interlocking software was developed, which had a variety of problems. Therefore, we developed a new parsing algorithm in order to recognize the variety of Launch Control ICD and verified that the algorithm operates normally by checking transmitting and receiving various message in conjunction with the fire control system.

Syntactic and Semantic Disambiguation for Interpretation of Numerals in the Information Retrieval (정보 검색을 위한 숫자의 해석에 관한 구문적.의미적 판별 기법)

  • Moon, Yoo-Jin
    • Journal of the Korea Society of Computer and Information
    • /
    • v.14 no.8
    • /
    • pp.65-71
    • /
    • 2009
  • Natural language processing is necessary in order to efficiently perform filtering tremendous information produced in information retrieval of world wide web. This paper suggested an algorithm for meaning of numerals in the text. The algorithm for meaning of numerals utilized context-free grammars with the chart parsing technique, interpreted affixes connected with the numerals and was designed to disambiguate their meanings systematically supported by the n-gram based words. And the algorithm was designed to use POS (part-of-speech) taggers, to automatically recognize restriction conditions of trigram words, and to gradually disambiguate the meaning of the numerals. This research performed experiment for the suggested system of the numeral interpretation. The result showed that the frequency-proportional method recognized the numerals with 86.3% accuracy and the condition-proportional method with 82.8% accuracy.

Document Filtering Algorithm for Efficient Preprocessing of XML Information Retrieval (XML 정보검색의 효율적 전처리를 위한 문서여과 알고리즘)

  • Kong Yong-Hae;Kim Myung-Sook
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.6 no.1
    • /
    • pp.1-11
    • /
    • 2005
  • The paper proposes a preprocessing method for efficient processing of XML queries in information retrieval with a large amount of XML documents. The conventional preprocessing methods filter out XML documents by parsing XML document for keyword of query or by comparing query signatures with signatures of XML document to be generated. But these methods are dependent on a query and are very in efficient for a large amount of XML documents. For this, we generate a universal DTD based on ontology of a domain. The universal DTD is applicable to the XML documents when they contain information of a same domain even when they have different structures and attributes. Then, using the universal DTD, we filter out the XML documents that are not bounded in the domain. We evaluate the performance of this method through experiments.

  • PDF

Generation of Video Clips Utilizing Shot Boundary Detection (샷 경계 검출을 이용한 영상 클립 생성)

  • Kim, Hyeok-Man;Cho, Seong-Kil
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.7 no.6
    • /
    • pp.582-592
    • /
    • 2001
  • Video indexing plays an important role in the applications such as digital video libraries or web VOD which archive large volume of digital videos. Video indexing is usually based on video segmentation. In this paper, we propose a software tool called V2Web Studio which can generate video clips utilizing shot boundary detection algorithm. With the V2Web Studio, the process of clip generation consists of the following four steps: 1) Automatic detection of shot boundaries by parsing the video, 2) Elimination of errors by manually verifying the results of the detection, 3) Building a modeling structure of logical hierarchy using the verified shots, and 4) Generating multiple video clips corresponding to each logically modeled segment. The aforementioned steps are performed by shot detector, shot verifier, video modeler and clip generator in the V2Web Studio respectively.

  • PDF

Accurate and Efficient Log Template Discovery Technique

  • Tak, Byungchul
    • Journal of the Korea Society of Computer and Information
    • /
    • v.23 no.10
    • /
    • pp.11-21
    • /
    • 2018
  • In this paper we propose a novel log template discovery algorithm which achieves high quality of discovered log templates through iterative log filtering technique. Log templates are the static string pattern of logs that are used to produce actual logs by inserting variable values during runtime. Identifying individual logs into their template category correctly enables us to conduct automated analysis using state-of-the-art machine learning techniques. Our technique looks at the group of logs column-wise and filters the logs that have the value of the highest proportion. We repeat this process per each column until we are left with highly homogeneous set of logs that most likely belong to the same log template category. Then, we determine which column is the static part and which is the variable part by vertically comparing all the logs in the group. This process repeats until we have discovered all the templates from given logs. Also, during this process we discover the custom patterns such as ID formats that are unique to the application. This information helps us quickly identify such strings in the logs as variable parts thereby further increasing the accuracy of the discovered log templates. Existing solutions suffer from log templates being too general or too specific because of the inability to detect custom patterns. Through extensive evaluations we have learned that our proposed method achieves 2 to 20 times better accuracy.

URL Signatures for Improving URL Normalization (URL 정규화 향상을 위한 URL 서명)

  • Soon, Lay-Ki;Lee, Sang-Ho
    • Journal of KIISE:Databases
    • /
    • v.36 no.2
    • /
    • pp.139-149
    • /
    • 2009
  • In the standard URL normalization mechanism, URLs are normalized syntactically by a set of predefined steps. In this paper, we propose to complement the standard URL normalization by incorporating the semantically meaningful metadata of the web pages. The metadata taken into consideration are the body texts and the page size of the web pages, which can be extracted during HTML parsing. The results from our first exploratory experiment indicate that the body texts are effective in identifying equivalent URLs. Hence, given a URL which has undergone the standard normalization, we construct its URL signature by hashing the body text of the associated web page using Message-Digest algorithm 5 in the second experiment. URLs which share identical signatures are considered to be equivalent in our scheme. The results in the second experiment show that our proposed URL signatures were able to further reduce redundant URLs by 32.94% in comparison with the standard URL normalization.

Semantic Scenes Classification of Sports News Video for Sports Genre Analysis (스포츠 장르 분석을 위한 스포츠 뉴스 비디오의 의미적 장면 분류)

  • Song, Mi-Young
    • Journal of Korea Multimedia Society
    • /
    • v.10 no.5
    • /
    • pp.559-568
    • /
    • 2007
  • Anchor-person scene detection is of significance for video shot semantic parsing and indexing clues extraction in content-based news video indexing and retrieval system. This paper proposes an efficient algorithm extracting anchor ranges that exist in sports news video for unit structuring of sports news. To detect anchor person scenes, first, anchor person candidate scene is decided by DCT coefficients and motion vector information in the MPEG4 compressed video. Then, from the candidate anchor scenes, image processing method is utilized to classify the news video into anchor-person scenes and non-anchor(sports) scenes. The proposed scheme achieves a mean precision and recall of 98% in the anchor-person scenes detection experiment.

  • PDF

Design and Implementation of Advanced Web Log Preprocess Algorithm for Rule based Web IDS (룰 기반 웹 IDS 시스템을 위한 효율적인 웹 로그 전처리 기법 설계 및 구현)

  • Lee, Hyung-Woo
    • Journal of Internet Computing and Services
    • /
    • v.9 no.5
    • /
    • pp.23-34
    • /
    • 2008
  • The number of web service user is increasing steadily as web-based service is offered in various form. But, web service has a vulnerability such as SQL Injection, Parameter Injection and DoS attack. Therefore, it is required for us to develop Web IDS system and additionally to offer Rule-base intrusion detection/response mechanism against those attacks. However, existing Web IDS system didn't correspond properly on recent web attack mechanism because they didn't including suitable pre-processing procedure on huge web log data. Therfore, we propose an efficient web log pre-processing mechanism for enhancing rule based detection and improving the performance of web IDS base attack response system. Proposed algorithm provides both a field unit parsing and a duplicated string elimination procedure on web log data. And it is also possible for us to construct improved web IDS system.

  • PDF