• 제목/요약/키워드: Big data Problem

Search Result 577, Processing Time 0.029 seconds

Research on the development of demand for medical and bio technology using big data (빅데이터 활용 의학·바이오 부문 사업화 가능 기술 연구)

  • Lee, Bongmun.;Nam, Gayoung;Kang, Byeong Chul;Kim, CheeYong
    • Journal of Korea Multimedia Society
    • /
    • v.25 no.2
    • /
    • pp.345-352
    • /
    • 2022
  • Conducting AI-based fusion business due to the increment of ICT fusion medical device has been expanded. In addition, AI-based medical devices help change existing medical system on treatment into the paradigm of customized treatment such as preliminary diagnosis and prevention. It will be generally promoted to the change of medical device industry. Although the current demand forecasting of medical biotechnology commercialization is based on the method of Delphi and AHP, there is a problem that it is difficult to have a generalization due to fluctuation results according to a pool of participants. Therefore, the purpose of the paper is to predict demand forecasting for identifying promising technology based on building up big data in medical biotechnology. The development method is to employ candidate technologies of keywords extracted from SCOPUS and to use word2vec for drawing analysis indicator, technological distance similarity, and recommended technological similarity of top-level items in order to achieve a reasonable result. In addition, the method builds up academic big data for 5 years (2016-2020) in order to commercialize technology excavation on demand perspective. Lastly, the paper employs global data studies in order to develop domestic and international demand for technology excavation in the medical biotechnology field.

For Improving Security Log Big Data Analysis Efficiency, A Firewall Log Data Standard Format Proposed (보안로그 빅데이터 분석 효율성 향상을 위한 방화벽 로그 데이터 표준 포맷 제안)

  • Bae, Chun-sock;Goh, Sung-cheol
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.30 no.1
    • /
    • pp.157-167
    • /
    • 2020
  • The big data and artificial intelligence technology, which has provided the foundation for the recent 4th industrial revolution, has become a major driving force in business innovation across industries. In the field of information security, we are trying to develop and improve an intelligent security system by applying these techniques to large-scale log data, which has been difficult to find effective utilization methods before. The quality of security log big data, which is the basis of information security AI learning, is an important input factor that determines the performance of intelligent security system. However, the difference and complexity of log data by various product has a problem that requires excessive time and effort in preprocessing big data with poor data quality. In this study, we research and analyze the cases related to log data collection of various firewall. By proposing firewall log data collection format standard, we hope to contribute to the development of intelligent security systems based on security log big data.

Subnet Selection Scheme based on probability to enhance process speed of Big Data (빅 데이터의 처리속도 향상을 위한 확률기반 서브넷 선택 기법)

  • Jeong, Yoon-Su;Kim, Yong-Tae;Park, Gil-Cheol
    • Journal of Digital Convergence
    • /
    • v.13 no.9
    • /
    • pp.201-208
    • /
    • 2015
  • With services such as SNS and facebook, Big Data popularize the use of small size such as micro blogs are increasing. However, the problem of accuracy and computational cost of the search result of big data of a small size is unresolved. In this paper, we propose a subnet selection techniques based probability to improve the browsing speed of the small size of the text information from big data environments, such as micro-blogs. The proposed method is to configure the subnets to give to the attribute information of the data increased the probability data search speed. In addition, the proposed method improves the accessibility of the data by processing a pair of the connection information between the probability of the data constituting the subnet to easily access the distributed data. Experimental results showed the proposed method is 6.8% higher detection rates than CELF algorithm, the average processing time was reduced by 8.2%.

Many-objective joint optimization for dependency-aware task offloading and service caching in mobile edge computing

  • Xiangyu Shi;Zhixia Zhang;Zhihua Cui;Xingjuan Cai
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.18 no.5
    • /
    • pp.1238-1259
    • /
    • 2024
  • Previous studies on joint optimization of computation offloading and service caching policies in Mobile Edge Computing (MEC) have often neglected the impact of dependency-aware subtasks, edge server resource constraints, and multiple users on policy formulation. To remedy this deficiency, this paper proposes a many-objective joint optimization dependency-aware task offloading and service caching model (MaJDTOSC). MaJDTOSC considers the impact of dependencies between subtasks on the joint optimization problem of task offloading and service caching in multi-user, resource-constrained MEC scenarios, and takes the task completion time, energy consumption, subtask hit rate, load variability, and storage resource utilization as optimization objectives. Meanwhile, in order to better solve MaJDTOSC, a many-objective evolutionary algorithm TSMSNSGAIII based on a three-stage mating selection strategy is proposed. Simulation results show that TSMSNSGAIII exhibits an excellent and stable performance in solving MaJDTOSC with different number of users setting and can converge faster. Therefore, it is believed that TSMSNSGAIII can provide appropriate sub-task offloading and service caching strategies in multi-user and resource-constrained MEC scenarios, which can greatly improve the system offloading efficiency and enhance the user experience.

Frequency and Social Network Analysis of the Bible Data using Big Data Analytics Tools R (빅데이터 분석도구 R을 이용한 성경 데이터의 빈도와 소셜 네트워크 분석)

  • Ban, ChaeHoon;Ha, JongSoo;Kim, Dong Hyun
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.24 no.2
    • /
    • pp.166-171
    • /
    • 2020
  • Big data processing technology that can store and analyze data and obtain new knowledge has been adjusted for importance in many fields of the society. Big data is emerging as an important problem in the field of information and communication technology, but the mind of continuous technology is rising. the R, a tool that can analyze big data, is a language and environment that enables information analysis of statistical bases. In this paper, we use this to analyze the Bible data. We analyze the four Gospels of the New Testament in the Bible. We collect the Bible data and perform filtering for analysis. The R is used to investigate the frequency of what text is distributed and analyze the Bible through social network analysis, in which words from a sentence are paired and analyzed between words for accurate data analysis.

Design of Extended Real-time Data Pipeline System Architecture (확장형 실시간 데이터 파이프라인 시스템 아키텍처 설계)

  • Shin, Hoseung;Kang, Sungwon;Lee, Jihyun
    • Journal of KIISE
    • /
    • v.42 no.8
    • /
    • pp.1010-1021
    • /
    • 2015
  • Big data systems are widely used to collect large-scale log data, so it is very important for these systems to operate with a high level of performance. However, the current Hadoop-based big data system architecture has a problem in that its performance is low as a result of redundant processing. This paper solves this problem by improving the design of the Hadoop system architecture. The proposed architecture uses the batch-based data collection of the existing architecture in combination with a single processing method. A high level of performance can be achieved by analyzing the collected data directly in memory to avoid redundant processing. The proposed architecture guarantees system expandability, which is an advantage of using the Hadoop architecture. This paper confirms that the proposed architecture is approximately 30% to 35% faster in analyzing and processing data than existing architectures and that it is also extendable.

Guidelines for big data projects in artificial intelligence mathematics education (인공지능 수학 교육을 위한 빅데이터 프로젝트 과제 가이드라인)

  • Lee, Junghwa;Han, Chaereen;Lim, Woong
    • The Mathematical Education
    • /
    • v.62 no.2
    • /
    • pp.289-302
    • /
    • 2023
  • In today's digital information society, student knowledge and skills to analyze big data and make informed decisions have become an important goal of school mathematics. Integrating big data statistical projects with digital technologies in high school <Artificial Intelligence> mathematics courses has the potential to provide students with a learning experience of high impact that can develop these essential skills. This paper proposes a set of guidelines for designing effective big data statistical project-based tasks and evaluates the tasks in the artificial intelligence mathematics textbook against these criteria. The proposed guidelines recommend that projects should: (1) align knowledge and skills with the national school mathematics curriculum; (2) use preprocessed massive datasets; (3) employ data scientists' problem-solving methods; (4) encourage decision-making; (5) leverage technological tools; and (6) promote collaborative learning. The findings indicate that few textbooks fully align with these guidelines, with most failing to incorporate elements corresponding to Guideline 2 in their project tasks. In addition, most tasks in the textbooks overlook or omit data preprocessing, either by using smaller datasets or by using big data without any form of preprocessing. This can potentially result in misconceptions among students regarding the nature of big data. Furthermore, this paper discusses the relevant mathematical knowledge and skills necessary for artificial intelligence, as well as the potential benefits and pedagogical considerations associated with integrating technology into big data tasks. This research sheds light on teaching mathematical concepts with machine learning algorithms and the effective use of technology tools in big data education.

A Trend Analysis on E-sports using Social Big Data

  • Kyoung Ah YEO;Min Soo KIM
    • Journal of Sport and Applied Science
    • /
    • v.8 no.1
    • /
    • pp.11-17
    • /
    • 2024
  • Purpose: The purpose of the study was to understand a trend of esports in terms of gamers' and fans' perceptions toward esports using social big data. Research design, data, and methodology: In this study, researchers first selected keywords related to esports. Then a total of 10,138 buzz data created at twitter, Facebook, news media, blogs, café and community between November 10, 2022 and November 19, 2023 were collected and analyzed with 'Textom', a big data solution. Results: The results of this study were as follows. Firstly, the news data's main articles were about competitions hosted by local governments and policies to revitalize the gaming industry. Secondly, As a result of esports analysis using Textom, there was a lot of interest in the adoption of the Hangzhou Asian Games as an official event and various esports competitions. As a result of the sentiment analysis, the positive content was related to the development potential of the esports industry, and the negative content was a discussion about the fundamental problem of whether esports is truly a sport. Thirdly, As a result of analyzing social big data on esports and the Olympics, there was hope that it would be adopted as an official event in the Olympics due to its adoption as an official event in the Hangzhou Asian Games. Conclusions: There was a positive opinion that the adoption of esports as an official Olympic event had positive content that could improve the quality of the game, and a negative opinion that games with actions that violate the Olympic spirit, such as murder and assault, should not be adopted as an official Olympic event. Further implications were discussed.

Panic Disorder Intelligent Health System based on IoT and Context-aware

  • Huan, Meng;Kang, Yun-Jeong;Lee, Sang-won;Choi, Dong-Oun
    • International journal of advanced smart convergence
    • /
    • v.10 no.2
    • /
    • pp.21-30
    • /
    • 2021
  • With the rapid development of artificial intelligence and big data, a lot of medical data is effectively used, and the diagnosis and analysis of diseases has entered the era of intelligence. With the increasing public health awareness, ordinary citizens have also put forward new demands for panic disorder health services. Specifically, people hope to predict the risk of panic disorder as soon as possible and grasp their own condition without leaving home. Against this backdrop, the smart health industry comes into being. In the Internet age, a lot of panic disorder health data has been accumulated, such as diagnostic records, medical record information and electronic files. At the same time, various health monitoring devices emerge one after another, enabling the collection and storage of personal daily health information at any time. How to use the above data to provide people with convenient panic disorder self-assessment services and reduce the incidence of panic disorder in China has become an urgent problem to be solved. In order to solve this problem, this research applies the context awareness to the automatic diagnosis of human diseases. While helping patients find diseases early and get treatment timely, it can effectively assist doctors in making correct diagnosis of diseases and reduce the probability of misdiagnosis and missed diagnosis.

Sequential Pattern Mining for Intrusion Detection System with Feature Selection on Big Data

  • Fidalcastro, A;Baburaj, E
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.11 no.10
    • /
    • pp.5023-5038
    • /
    • 2017
  • Big data is an emerging technology which deals with wide range of data sets with sizes beyond the ability to work with software tools which is commonly used for processing of data. When we consider a huge network, we have to process a large amount of network information generated, which consists of both normal and abnormal activity logs in large volume of multi-dimensional data. Intrusion Detection System (IDS) is required to monitor the network and to detect the malicious nodes and activities in the network. Massive amount of data makes it difficult to detect threats and attacks. Sequential Pattern mining may be used to identify the patterns of malicious activities which have been an emerging popular trend due to the consideration of quantities, profits and time orders of item. Here we propose a sequential pattern mining algorithm with fuzzy logic feature selection and fuzzy weighted support for huge volumes of network logs to be implemented in Apache Hadoop YARN, which solves the problem of speed and time constraints. Fuzzy logic feature selection selects important features from the feature set. Fuzzy weighted supports provide weights to the inputs and avoid multiple scans. In our simulation we use the attack log from NS-2 MANET environment and compare the proposed algorithm with the state-of-the-art sequential Pattern Mining algorithm, SPADE and Support Vector Machine with Hadoop environment.