• Title/Summary/Keyword: Big data Processing

Search Result 1,063, Processing Time 0.031 seconds

Using IoT and Apache Spark Analysis Technique to Monitoring Architecture Model for Fruit Harvest Region (IoT 기반 Apache Spark 분석기법을 이용한 과수 수확 불량 영역 모니터링 아키텍처 모델)

  • Oh, Jung Won;Kim, Hangkon
    • Smart Media Journal
    • /
    • v.6 no.4
    • /
    • pp.58-64
    • /
    • 2017
  • Modern society is characterized by rapid increase in world population, aging of the rural population, decrease of cultivation area due to industrialization. The food problem is becoming an important issue with the farmers and becomes rural. Recently, the researches about the field of the smart farm are actively carried out to increase the profit of the rural area. The existing smart farm researches mainly monitor the cultivation environment of the crops in the greenhouse, another way like in the case of poor quality t is being studied that the system to control cultivation environmental factors is automatically activated to keep the cultivation environment of crops in optimum conditions. The researches focus on the crops cultivated indoors, and there are not many studies applied to the cultivation environment of crops grown outside. In this paper, we propose a method to improve the harvestability of poor areas by monitoring the areas with bad harvests by using big data analysis, by precisely predicting the harvest timing of fruit trees growing in orchards. Factors besides for harvesting include fruit color information and fruit weight information We suggest that a harvest correlation factor data collected in real time. It is analyzed using the Apache Spark engine. The Apache Spark engine has excellent performance in real-time data analysis as well as high capacity batch data analysis. User device receiving service supports PC user and smartphone users. A sensing data receiving device purpose Arduino, because it requires only simple processing to receive a sensed data and transmit it to the server. It regulates a harvest time of fruit which produces a good quality fruit, it is needful to determine a poor harvest area or concentrate a bad area. In this paper, we also present an architectural model to determine the bad areas of fruit harvest using strong data analysis.

Artificial Intelligence-based Security Control Construction and Countermeasures (인공지능기반 보안관제 구축 및 대응 방안)

  • Hong, Jun-Hyeok;Lee, Byoung Yup
    • The Journal of the Korea Contents Association
    • /
    • v.21 no.1
    • /
    • pp.531-540
    • /
    • 2021
  • As cyber attacks and crimes increase exponentially and hacking attacks become more intelligent and advanced, hacking attack methods and routes are evolving unpredictably and in real time. In order to reinforce the enemy's responsiveness, this study aims to propose a method for developing an artificial intelligence-based security control platform by building a next-generation security system using artificial intelligence to respond by self-learning, monitoring abnormal signs and blocking attacks.The artificial intelligence-based security control platform should be developed as the basis for data collection, data analysis, next-generation security system operation, and security system management. Big data base and control system, data collection step through external threat information, data analysis step of pre-processing and formalizing the collected data to perform positive/false detection and abnormal behavior analysis through deep learning-based algorithm, and analyzed data Through the operation of a security system of prevention, control, response, analysis, and organic circulation structure, the next generation security system to increase the scope and speed of handling new threats and to reinforce the identification of normal and abnormal behaviors, and management of the security threat response system, Harmful IP management, detection policy management, security business legal system management. Through this, we are trying to find a way to comprehensively analyze vast amounts of data and to respond preemptively in a short time.

Building a Korean Sentiment Lexicon Using Collective Intelligence (집단지성을 이용한 한글 감성어 사전 구축)

  • An, Jungkook;Kim, Hee-Woong
    • Journal of Intelligence and Information Systems
    • /
    • v.21 no.2
    • /
    • pp.49-67
    • /
    • 2015
  • Recently, emerging the notion of big data and social media has led us to enter data's big bang. Social networking services are widely used by people around the world, and they have become a part of major communication tools for all ages. Over the last decade, as online social networking sites become increasingly popular, companies tend to focus on advanced social media analysis for their marketing strategies. In addition to social media analysis, companies are mainly concerned about propagating of negative opinions on social networking sites such as Facebook and Twitter, as well as e-commerce sites. The effect of online word of mouth (WOM) such as product rating, product review, and product recommendations is very influential, and negative opinions have significant impact on product sales. This trend has increased researchers' attention to a natural language processing, such as a sentiment analysis. A sentiment analysis, also refers to as an opinion mining, is a process of identifying the polarity of subjective information and has been applied to various research and practical fields. However, there are obstacles lies when Korean language (Hangul) is used in a natural language processing because it is an agglutinative language with rich morphology pose problems. Therefore, there is a lack of Korean natural language processing resources such as a sentiment lexicon, and this has resulted in significant limitations for researchers and practitioners who are considering sentiment analysis. Our study builds a Korean sentiment lexicon with collective intelligence, and provides API (Application Programming Interface) service to open and share a sentiment lexicon data with the public (www.openhangul.com). For the pre-processing, we have created a Korean lexicon database with over 517,178 words and classified them into sentiment and non-sentiment words. In order to classify them, we first identified stop words which often quite likely to play a negative role in sentiment analysis and excluded them from our sentiment scoring. In general, sentiment words are nouns, adjectives, verbs, adverbs as they have sentimental expressions such as positive, neutral, and negative. On the other hands, non-sentiment words are interjection, determiner, numeral, postposition, etc. as they generally have no sentimental expressions. To build a reliable sentiment lexicon, we have adopted a concept of collective intelligence as a model for crowdsourcing. In addition, a concept of folksonomy has been implemented in the process of taxonomy to help collective intelligence. In order to make up for an inherent weakness of folksonomy, we have adopted a majority rule by building a voting system. Participants, as voters were offered three voting options to choose from positivity, negativity, and neutrality, and the voting have been conducted on one of the largest social networking sites for college students in Korea. More than 35,000 votes have been made by college students in Korea, and we keep this voting system open by maintaining the project as a perpetual study. Besides, any change in the sentiment score of words can be an important observation because it enables us to keep track of temporal changes in Korean language as a natural language. Lastly, our study offers a RESTful, JSON based API service through a web platform to make easier support for users such as researchers, companies, and developers. Finally, our study makes important contributions to both research and practice. In terms of research, our Korean sentiment lexicon plays an important role as a resource for Korean natural language processing. In terms of practice, practitioners such as managers and marketers can implement sentiment analysis effectively by using Korean sentiment lexicon we built. Moreover, our study sheds new light on the value of folksonomy by combining collective intelligence, and we also expect to give a new direction and a new start to the development of Korean natural language processing.

ACCELERATION OF MACHINE LEARNING ALGORITHMS BY TCHEBYCHEV ITERATION TECHNIQUE

  • LEVIN, MIKHAIL P.
    • Journal of the Korean Society for Industrial and Applied Mathematics
    • /
    • v.22 no.1
    • /
    • pp.15-28
    • /
    • 2018
  • Recently Machine Learning algorithms are widely used to process Big Data in various applications and a lot of these applications are executed in run time. Therefore the speed of Machine Learning algorithms is a critical issue in these applications. However the most of modern iteration Machine Learning algorithms use a successive iteration technique well-known in Numerical Linear Algebra. But this technique has a very low convergence, needs a lot of iterations to get solution of considering problems and therefore a lot of time for processing even on modern multi-core computers and clusters. Tchebychev iteration technique is well-known in Numerical Linear Algebra as an attractive candidate to decrease the number of iterations in Machine Learning iteration algorithms and also to decrease the running time of these algorithms those is very important especially in run time applications. In this paper we consider the usage of Tchebychev iterations for acceleration of well-known K-Means and SVM (Support Vector Machine) clustering algorithms in Machine Leaning. Some examples of usage of our approach on modern multi-core computers under Apache Spark framework will be considered and discussed.

Performance Consequences of Convergence and Divergence in Strategic Positioning

  • Park, Kyung-Min
    • 한국산학경영학회:학술대회논문집
    • /
    • 2005.11a
    • /
    • pp.73-94
    • /
    • 2005
  • This paper investigates the performance consequence of strategic changes when firms move closer to or further away from other firms in the industry. The study suggests a theoretical framework and hypotheses on the effect of strategic convergence and divergence on performance, and tests hypotheses with firm-level longitudinal data on the U.S. food processing industry during the period of 1985-2000. The study shows that strategic divergence is negatively related to performance, and that organizational size and firm-specific uncertainty significantly influence the effect of strategic convergence and divergence on financial performance. Particularly, high uncertainty seems to be conducive to financial performance improvement for organizations undergoing significant strategic changes converging toward other competitors. On the other hand, big organizational size seems to be beneficial for finns implementing strategic changesdiverging from other competitors.

  • PDF

Technology Trends of AI for Big Data Knowledge Processing (빅데이터 지식처리 인공지능 기술동향)

  • Lee, H.J.;Ryu, P.M.;Lim, S.J.;Jang, M.K.;Kim, H.K.
    • Electronics and Telecommunications Trends
    • /
    • v.29 no.4
    • /
    • pp.30-38
    • /
    • 2014
  • 최근의 플랫폼 기술동향은 웹 기반 혹은 단순 의사소통이 가능한 모바일 플랫폼에서 빅데이터와 인공지능기술이 접목되면서 심층 질의응답이 가능한 차세대 지능형 지식처리 플랫폼으로의 진화가 진행 중이다. 선진국에서는 국가 차원 혹은 글로벌 기업의 주도하에 대형 장기 프로젝트가 진행 중이다. 국가 주도의 프로젝트로는 미국의 PAL, 유럽의 Human Brain, 일본의 Todai 프로젝트가 대표적인 예이며, 글로벌 기업의 경우는 IBM의 Watson, Google의 Knowledge Graph, Apple의 Sir가 대표적인 예이다. 본고에서는 차세대 지능형 플랫폼의 핵심기술인 인간과 기계의 지식소통을 위한 빅데이터 기반의 지식처리 인공지능 소프트웨어 기술의 개념과 국내외 기술 및 산업, 지식재산권 동향 등을 살펴보고 산업계 활용방안 및 발전방향에 대해 논하고자 한다.

  • PDF

Design and Implementation of Hadoop-based Platform "Textom" for Processing Big-data (하둡 기반 빅데이터 수집 및 처리를 위한 플랫폼 설계 및 구현)

  • Son, ki-jun;Cho, in-ho;Kim, chan-woo;Jun, chae-nam
    • Proceedings of the Korea Contents Association Conference
    • /
    • 2015.05a
    • /
    • pp.297-298
    • /
    • 2015
  • 빅데이터 처리를 위한 소프트웨어 시스템을 구축하기 위하여 필요한 대표적인 기술 중 하나가 데이터의 수집 및 분석이다. 데이터 수집은 서비스를 제공하기 위한 분석의 기초 작업으로 분석 인프라를 구축하는 작업에 매우 중요하다. 본 논문은 한국어 기반 빅데이터 처리를 위하여 웹과 SNS상의 데이터 수집 어플리케이션 및 저장과 분석을 위한 플랫폼을 제공한다. 해당 플랫폼은 하둡(Hadoop) 기반으로 동작을 하며 비동기적으로 데이터를 수집하고, 수집된 데이터를 하둡에 저장하게 되며, 저장된 데이터를 분석한 후 분석결과에 대한 시각화 결과를 제공한다. 구현된 빅데이터 플랫폼 텍스톰은 데이터 수집 및 분석가를 위한 유용한 시스템이 될 것으로 기대가 된다. 특히 본 논문에서는 모든 구현을 오픈소스 소프트웨어에 기반하여 수행했으며, 웹 환경에서 데이터 수집 및 분석이 가능하도록 구현하였다.

  • PDF

Drone Simulation Technologies (드론 시뮬레이션 기술)

  • Lee, S.J.;Yang, J.G.;Lee, B.S.
    • Electronics and Telecommunications Trends
    • /
    • v.35 no.4
    • /
    • pp.81-90
    • /
    • 2020
  • The use of machine learning technologies such as deep and reinforcement learning has proliferated in various domains with the advancement of deep neural network studies. To make the learning successful, both big data acquisition and fast processing are required. However, for some physical world applications such as autonomous drone flight, it is difficult to achieve efficient learning because learning with a premature A.I. is dangerous, cost-ineffective, and time-consuming. To solve these problems, simulation-based approaches can be considered. In this study, we analyze recent trends in drone simulation technologies and compare their features. Subsequently, we introduce Octopus, which is a highly precise and scalable drone simulator being developed by ETRI.

Monitoring of Mechanical Seal Failure with Artificial Neural Network (신경회로망을 이용한 미케니컬 실의 이상상태 감시)

  • Lee, W.K.;Lim, S.J.;Namgung, S.
    • Journal of the Korean Society for Precision Engineering
    • /
    • v.12 no.12
    • /
    • pp.30-37
    • /
    • 1995
  • The mechanical seals, which are installed in rotating machines like pump and compressor, are gengrally used as sealing devices in the many fields of industries. The failure of mechanical seals such as leakage,fast and severe wear, excessive torque, and squeaking results in big problems. To monitor the failure of mechanical seals and to propose the proper monitoring techniques with artificial neural network, sliding wear experiments were conducted. Torque and temperature of the mechanical seals were measured during experiments. Optical microstructure was observed for the wear processing after every 10 minute sliding at rotation speed of 1750 rpm and scanning electron microscopy was also observed. During the experiment, the variation of torque and temperature that meant an abnormal phenomenon, was observed. That experimental data recorded were applied to the developed monitoring system with artificial neural network. This study concludes that torque and temperature of mechanical seals wil be used to identify and to monitor the condition of sliding motion of mechanical seals. An availability to monitor the mechanical seal failure with artificial neural network was confirmed.

  • PDF

Design of Incremental FCM-based RBF Neural Networks Pattern Classifier for Processing Big Data (빅 데이터 처리를 위한 증분형 FCM 기반 RBF Neural Networks 패턴 분류기 설계)

  • Lee, Seung-Cheol;Oh, Sung-Kwun;Roh, Seok-Beom
    • Proceedings of the KIEE Conference
    • /
    • 2015.07a
    • /
    • pp.1343-1344
    • /
    • 2015
  • 본 연구에서는 증분형 FCM(Incremental Fuzzy C-Means: Incremental FCM) 클러스터링 알고리즘을 기반으로 방사형 기저함수 신경회로망(Radial Basis Function Neural Networks: RBFNN) 패턴 분류기를 설계한다. 방사형 기저함수 신경회로망은 조건부에서 가우시안 함수 또는 FCM을 사용하여 적합도를 구하였지만, 제안된 분류기에서는 빅 데이터간의 적합도를 구하기 위해 증분형 FCM을 사용한다. 또한, 빅 데이터를 학습하기 위해 결론부에서 재귀최소자승법(Recursive Least Square Estimation: RLSE)을 사용하여 다항식 계수를 추정한다. 마지막으로 추론부에서는 증분형 FCM에서 구한 적합도와 재귀최소자승법으로 구한 다항식을 이용하여 최종 출력을 구한다.

  • PDF