• Title/Summary/Keyword: 그래프 데이터 셋

Search Result 50, Processing Time 0.032 seconds

RDBMS Based Efficient Method for Shortest Path Searching Over Large Graphs Using K-degree Index Table (대용량 그래프에서 k-차수 인덱스 테이블을 이용한 RDBMS 기반의 효율적인 최단 경로 탐색 기법)

  • Hong, Jihye;Han, Yongkoo;Lee, Young-Koo
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.3 no.5
    • /
    • pp.179-186
    • /
    • 2014
  • Current networks such as social network, web page link, traffic network are big data which have the large numbers of nodes and edges. Many applications such as social network services and navigation systems use these networks. Since big networks are not fit into the memory, existing in-memory based analysis techniques cannot provide high performance. Frontier-Expansion-Merge (FEM) framework for graph search operations using three corresponding operators in the relational database (RDB) context. FEM exploits an index table that stores pre-computed partial paths for efficient shortest path discovery. However, the index table of FEM has low hit ratio because the indices are determined by distances of indices rather than the possibility of containing a shortest path. In this paper, we propose an method that construct index table using high degree nodes having high hit ratio for efficient shortest path discovery. We experimentally verify that our index technique can support shortest path discovery efficiently in real-world datasets.

Graph Transformer Network based Wireless Network Intrusion Detection System (Graph Transformer Network 기반 무선 네트워크 침입 탐지 시스템)

  • Seok-Won Hong;Jin-Seong Kim;Min-Jae Kim;Seok-Hwan Choi
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2024.05a
    • /
    • pp.882-884
    • /
    • 2024
  • 수많은 무선 네트워크 서비스의 등장과 함께 무선 네트워크를 대상으로 한 공격이 증가하고 있다. 이러한 공격을 탐지하기 위해 최근 많은 연구가 진행되고 있다. 특히 네트워크의 복잡한 연결 구조와 패턴을 효율적으로 분석할 수 있는 그래프 기반 인공지능 모델이 적용된 네트워크 침입 탐지 시스템(Network Intrusion Detection System, NIDS)에 관한 다양한 연구가 진행되고 있다. 이러한 배경을 바탕으로 본 논문에서는 무선 네트워크를 대상으로 한 공격의 정확하고 신속한 탐지를 위한 Graph Transformer Network(GTN) 기반 네트워크 침입 탐지 시스템을 제안하고 AWID3 데이터셋을 이용한 실험을 통해 GTN 기반 NIDS의 우수성을 검증한다.

Transitive Similarity Evaluation Model for Improving Sparsity in Collaborative Filtering (협업필터링의 희박 행렬 문제를 위한 이행적 유사도 평가 모델)

  • Bae, Eun-Young;Yu, Seok-Jong
    • The Journal of Korean Institute of Information Technology
    • /
    • v.16 no.12
    • /
    • pp.109-114
    • /
    • 2018
  • Collaborative filtering has been widely utilized in recommender systems as typical algorithm for outstanding performance. Since it depends on item rating history structurally, The more sparse rating matrix is, the lower its recommendation accuracy is, and sometimes it is totally useless. Variety of hybrid approaches have tried to combine collaborative filtering and content-based method for improving the sparsity issue in rating matrix. In this study, a new method is suggested for the same purpose, but with different perspective, it deals with no-match situation in person-person similarity evaluation. This method is called the transitive similarity model because it is based on relation graph of people, and it compares recommendation accuracy by applying to Movielens open dataset.

Building change detection in high spatial resolution images using deep learning and graph model (딥러닝과 그래프 모델을 활용한 고해상도 영상의 건물 변화탐지)

  • Park, Seula;Song, Ahram
    • Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
    • /
    • v.40 no.3
    • /
    • pp.227-237
    • /
    • 2022
  • The most critical factors for detecting changes in very high-resolution satellite images are building positional inconsistencies and relief displacements caused by satellite side-view. To resolve the above problems, additional processing using a digital elevation model and deep learning approach have been proposed. Unfortunately, these approaches are not sufficiently effective in solving these problems. This study proposed a change detection method that considers both positional and topology information of buildings. Mask R-CNN (Region-based Convolutional Neural Network) was trained on a SpaceNet building detection v2 dataset, and the central points of each building were extracted as building nodes. Then, triangulated irregular network graphs were created on building nodes from temporal images. To extract the area, where there is a structural difference between two graphs, a change index reflecting the similarity of the graphs and differences in the location of building nodes was proposed. Finally, newly changed or deleted buildings were detected by comparing the two graphs. Three pairs of test sites were selected to evaluate the proposed method's effectiveness, and the results showed that changed buildings were detected in the case of side-view satellite images with building positional inconsistencies.

Processing large-scale data with Apache Spark (Apache Spark를 활용한 대용량 데이터의 처리)

  • Ko, Seyoon;Won, Joong-Ho
    • The Korean Journal of Applied Statistics
    • /
    • v.29 no.6
    • /
    • pp.1077-1094
    • /
    • 2016
  • Apache Spark is a fast and general-purpose cluster computing package. It provides a new abstraction named resilient distributed dataset, which is capable of support for fault tolerance while keeping data in memory. This type of abstraction results in a significant speedup compared to legacy large-scale data framework, MapReduce. In particular, Spark framework is suitable for iterative machine learning applications such as logistic regression and K-means clustering, and interactive data querying. Spark also supports high level libraries for various applications such as machine learning, streaming data processing, database querying and graph data mining thanks to its versatility. In this work, we introduce the concept and programming model of Spark as well as show some implementations of simple statistical computing applications. We also review the machine learning package MLlib, and the R language interface SparkR.

Personalized Session-based Recommendation for Set-Top Box Audience Targeting (셋톱박스 오디언스 타겟팅을 위한 세션 기반 개인화 추천 시스템 개발)

  • Jisoo Cha;Koosup Jeong;Wooyoung Kim;Jaewon Yang;Sangduk Baek;Wonjun Lee;Seoho Jang;Taejoon Park;Chanwoo Jeong;Wooju Kim
    • Journal of Intelligence and Information Systems
    • /
    • v.29 no.2
    • /
    • pp.323-338
    • /
    • 2023
  • TV advertising with deep analysis of watching pattern of audiences is important to set-top box audience targeting. Applying session-based recommendation model(SBR) to internet commercial, or recommendation based on searching history of user showed its effectiveness in previous studies, but applying SBR to the TV advertising was difficult in South Korea due to data unavailabilities. Also, traditional SBR has limitations for dealing with user preferences, especially in data with user identification information. To tackle with these problems, we first obtain set-top box data from three major broadcasting companies in South Korea(SKB, KT, LGU+) through collaboration with Korea Broadcast Advertising Corporation(KOBACO), and this data contains of watching sequence of 4,847 anonymized users for 6 month respectively. Second, we develop personalized session-based recommendation model to deal with hierarchical data of user-session-item. Experiments conducted on set-top box audience dataset and two other public dataset for validation. In result, our proposed model outperformed baseline model in some criteria.

Document Summarization Considering Entailment Relation between Sentences (문장 수반 관계를 고려한 문서 요약)

  • Kwon, Youngdae;Kim, Noo-ri;Lee, Jee-Hyong
    • Journal of KIISE
    • /
    • v.44 no.2
    • /
    • pp.179-185
    • /
    • 2017
  • Document summarization aims to generate a summary that is consistent and contains the highly related sentences in a document. In this study, we implemented for document summarization that extracts highly related sentences from a whole document by considering both similarities and entailment relations between sentences. Accordingly, we proposed a new algorithm, TextRank-NLI, which combines a Recurrent Neural Network based Natural Language Inference model and a Graph-based ranking algorithm used in single document extraction-based summarization task. In order to evaluate the performance of the new algorithm, we conducted experiments using the same datasets as used in TextRank algorithm. The results indicated that TextRank-NLI showed 2.3% improvement in performance, as compared to TextRank.

Pyramid Feature Compression with Inter-Level Feature Restoration-Prediction Network (계층 간 특징 복원-예측 네트워크를 통한 피라미드 특징 압축)

  • Kim, Minsub;Sim, Donggyu
    • Journal of Broadcast Engineering
    • /
    • v.27 no.3
    • /
    • pp.283-294
    • /
    • 2022
  • The feature map used in the network for deep learning generally has larger data than the image and a higher compression rate than the image compression rate is required to transmit the feature map. This paper proposes a method for transmitting a pyramid feature map with high compression rate, which is used in a network with an FPN structure that has robustness to object size in deep learning-based image processing. In order to efficiently compress the pyramid feature map, this paper proposes a structure that predicts a pyramid feature map of a level that is not transmitted with pyramid feature map of some levels that transmitted through the proposed prediction network to efficiently compress the pyramid feature map and restores compression damage through the proposed reconstruction network. Suggested mAP, the performance of object detection for the COCO data set 2017 Train images of the proposed method, showed a performance improvement of 31.25% in BD-rate compared to the result of compressing the feature map through VTM12.0 in the rate-precision graph, and compared to the method of performing compression through PCA and DeepCABAC, the BD-rate improved by 57.79%.

Power-Laws Exponents of the Domestic AS-level Internet Topology based on the Valley-free BGP Routing Policy (BGP 밸리-프리 라우팅 정책에 기반한 국내 AS 레벨 인터넷 토폴로지의 파워-로 지수)

  • Kang, Koo-Hong
    • Journal of Internet Computing and Services
    • /
    • v.11 no.4
    • /
    • pp.41-49
    • /
    • 2010
  • Faloustsos et al.[1,2] showed successfully that the power-laws describe the heavy-tailed distributions of the AS level Internet topology properties such as node degree. This result allows us to represent the characteristics of AS-level Internet topology using some power-law exponents with elegant and simple. In this paper, we obtained the power-law exponents of the domestic AS-level Internet topology properties - the node degree, the number of pairs within hops, and eigenvalues of the graph - based on the valley-free BGP routing policy. We used the real data sets from UCLA IRL laboratory, and showed that these power-laws fit the real data pretty well resulting in correlation coefficient of 90.7%, 96.5%, and 97%, respectively. In particular, rounding the effective diameter to three, approximately 91% of the pairs of nodes are within this distance; that is, we might conclude our topology is pretty well organized.

Emoticon by Emotions: The Development of an Emoticon Recommendation System Based on Consumer Emotions (Emoticon by Emotions: 소비자 감성 기반 이모티콘 추천 시스템 개발)

  • Kim, Keon-Woo;Park, Do-Hyung
    • Journal of Intelligence and Information Systems
    • /
    • v.24 no.1
    • /
    • pp.227-252
    • /
    • 2018
  • The evolution of instant communication has mirrored the development of the Internet and messenger applications are among the most representative manifestations of instant communication technologies. In messenger applications, senders use emoticons to supplement the emotions conveyed in the text of their messages. The fact that communication via messenger applications is not face-to-face makes it difficult for senders to communicate their emotions to message recipients. Emoticons have long been used as symbols that indicate the moods of speakers. However, at present, emoticon-use is evolving into a means of conveying the psychological states of consumers who want to express individual characteristics and personality quirks while communicating their emotions to others. The fact that companies like KakaoTalk, Line, Apple, etc. have begun conducting emoticon business and sales of related content are expected to gradually increase testifies to the significance of this phenomenon. Nevertheless, despite the development of emoticons themselves and the growth of the emoticon market, no suitable emoticon recommendation system has yet been developed. Even KakaoTalk, a messenger application that commands more than 90% of domestic market share in South Korea, just grouped in to popularity, most recent, or brief category. This means consumers face the inconvenience of constantly scrolling around to locate the emoticons they want. The creation of an emoticon recommendation system would improve consumer convenience and satisfaction and increase the sales revenue of companies the sell emoticons. To recommend appropriate emoticons, it is necessary to quantify the emotions that the consumer sees and emotions. Such quantification will enable us to analyze the characteristics and emotions felt by consumers who used similar emoticons, which, in turn, will facilitate our emoticon recommendations for consumers. One way to quantify emoticons use is metadata-ization. Metadata-ization is a means of structuring or organizing unstructured and semi-structured data to extract meaning. By structuring unstructured emoticon data through metadata-ization, we can easily classify emoticons based on the emotions consumers want to express. To determine emoticons' precise emotions, we had to consider sub-detail expressions-not only the seven common emotional adjectives but also the metaphorical expressions that appear only in South Korean proved by previous studies related to emotion focusing on the emoticon's characteristics. We therefore collected the sub-detail expressions of emotion based on the "Shape", "Color" and "Adumbration". Moreover, to design a highly accurate recommendation system, we considered both emotion-technical indexes and emoticon-emotional indexes. We then identified 14 features of emoticon-technical indexes and selected 36 emotional adjectives. The 36 emotional adjectives consisted of contrasting adjectives, which we reduced to 18, and we measured the 18 emotional adjectives using 40 emoticon sets randomly selected from the top-ranked emoticons in the KakaoTalk shop. We surveyed 277 consumers in their mid-twenties who had experience purchasing emoticons; we recruited them online and asked them to evaluate five different emoticon sets. After data acquisition, we conducted a factor analysis of emoticon-emotional factors. We extracted four factors that we named "Comic", Softness", "Modernity" and "Transparency". We analyzed both the relationship between indexes and consumer attitude and the relationship between emoticon-technical indexes and emoticon-emotional factors. Through this process, we confirmed that the emoticon-technical indexes did not directly affect consumer attitudes but had a mediating effect on consumer attitudes through emoticon-emotional factors. The results of the analysis revealed the mechanism consumers use to evaluate emoticons; the results also showed that consumers' emoticon-technical indexes affected emoticon-emotional factors and that the emoticon-emotional factors affected consumer satisfaction. We therefore designed the emoticon recommendation system using only four emoticon-emotional factors; we created a recommendation method to calculate the Euclidean distance from each factors' emotion. In an attempt to increase the accuracy of the emoticon recommendation system, we compared the emotional patterns of selected emoticons with the recommended emoticons. The emotional patterns corresponded in principle. We verified the emoticon recommendation system by testing prediction accuracy; the predictions were 81.02% accurate in the first result, 76.64% accurate in the second, and 81.63% accurate in the third. This study developed a methodology that can be used in various fields academically and practically. We expect that the novel emoticon recommendation system we designed will increase emoticon sales for companies who conduct business in this domain and make consumer experiences more convenient. In addition, this study served as an important first step in the development of an intelligent emoticon recommendation system. The emotional factors proposed in this study could be collected in an emotional library that could serve as an emotion index for evaluation when new emoticons are released. Moreover, by combining the accumulated emotional library with company sales data, sales information, and consumer data, companies could develop hybrid recommendation systems that would bolster convenience for consumers and serve as intellectual assets that companies could strategically deploy.