• Title/Summary/Keyword: statistical data processing

Search Result 685, Processing Time 0.026 seconds

Segmentation-based Signal Processing Algorithm for Vehicle Detection (차량검지를 위한 세그먼트에 기반을 둔 신호처리 알고리즘)

  • Ko, Ki-Won;Woo, Kwang-Joon
    • Proceedings of the KIEE Conference
    • /
    • 2005.10b
    • /
    • pp.306-308
    • /
    • 2005
  • The vehicle detection method using pulse radar has the advantage of maintenance in comparison with loop detection method. We have the information about the vehicle being and position by dividing the signals into sectors in accordance with SSC method, and by applying the discriminant function based on stochastical data. We also reduce the signal processing time.

  • PDF

Big Data Platform Based on Hadoop and Application to Weight Estimation of FPSO Topside

  • Kim, Seong-Hoon;Roh, Myung-Il;Kim, Ki-Su;Oh, Min-Jae
    • Journal of Advanced Research in Ocean Engineering
    • /
    • v.3 no.1
    • /
    • pp.32-40
    • /
    • 2017
  • Recently, the amount of data to be processed and the complexity thereof have been increasing due to the development of information and communication technology, and industry's interest in such big data is increasing day by day. In the shipbuilding and offshore industry also, there is growing interest in the effective utilization of data, since various and vast amounts of data are being generated in the process of design, production, and operation. In order to effectively utilize big data in the shipbuilding and offshore industry, it is necessary to store and process large amounts of data. In this study, it was considered efficient to apply Hadoop and R, which are mostly used in big data related research. Hadoop is a framework for storing and processing big data. It provides the Hadoop Distributed File System (HDFS) for storing big data, and the MapReduce function for processing. Meanwhile, R provides various data analysis techniques through the language and environment for statistical calculation and graphics. While Hadoop makes it is easy to handle big data, it is difficult to finely process data; and although R has advanced analysis capability, it is difficult to use to process large data. This study proposes a big data platform based on Hadoop for applications in the shipbuilding and offshore industry. The proposed platform includes the existing data of the shipyard, and makes it possible to manage and process the data. To check the applicability of the platform, it is applied to estimate the weights of offshore structure topsides. In this study, we store data of existing FPSOs in Hadoop-based Hortonworks Data Platform (HDP), and perform regression analysis using RHadoop. We evaluate the effectiveness of large data processing by RHadoop by comparing the results of regression analysis and the processing time, with the results of using the conventional weight estimation program.

Online abnormal events detection with online support vector machine (온라인 서포트벡터기계를 이용한 온라인 비정상 사건 탐지)

  • Park, Hye-Jung
    • Journal of the Korean Data and Information Science Society
    • /
    • v.22 no.2
    • /
    • pp.197-206
    • /
    • 2011
  • The ability to detect online abnormal events in signals is essential in many real-world signal processing applications. In order to detect abnormal events, previously known algorithms require an explicit signal statistical model, and interpret abnormal events as statistical model abrupt changes. In general, maximum likelihood and Bayesian estimation theory to estimate well as detection methods have been used. However, the above-mentioned methods for robust and tractable model, it is not easy to estimate. More freedom to estimate how the model is needed. In this paper, we investigate a machine learning, descriptor-based approach that does not require a explicit descriptors statistical model, based on support vector machines are known to be robust statistical models and a sequential optimal algorithm online support vector machine is introduced.

Reverse Engineering of a Gene Regulatory Network from Time-Series Data Using Mutual Information

  • Barman, Shohag;Kwon, Yung-Keun
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2014.11a
    • /
    • pp.849-852
    • /
    • 2014
  • Reverse engineering of gene regulatory network is a challenging task in computational biology. To detect a regulatory relationship among genes from time series data is called reverse engineering. Reverse engineering helps to discover the architecture of the underlying gene regulatory network. Besides, it insights into the disease process, biological process and drug discovery. There are many statistical approaches available for reverse engineering of gene regulatory network. In our paper, we propose pairwise mutual information for the reverse engineering of a gene regulatory network from time series data. Firstly, we create random boolean networks by the well-known $Erd{\ddot{o}}s-R{\acute{e}}nyi$ model. Secondly, we generate artificial time series data from that network. Then, we calculate pairwise mutual information for predicting the network. We implement of our system on java platform. To visualize the random boolean network graphically we use cytoscape plugins 2.8.0.

Hyper-encryption Scheme for Data Confidentiality in Wireless Broadband (WiBro) Networks

  • Hamid, Abdul;Hong, Choong-Seon
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2007.05a
    • /
    • pp.1096-1097
    • /
    • 2007
  • We address the data confidentiality for wireless broadband (WiBro) networks. In WiBro, as the channel is wireless in nature, it suffers from passive and active attack. Passive attack, for example is to decrypt traffic based on statistical analysis and active attack is to modify traffic or inject new traffic from unauthorized mobile stations. Due to high mobility, frequent session key distribution is a bottleneck for the mobile stations. In aspect of WiBro, there is a communication between mobile station to base station, and also in mobile station to mobile station. It is expected to ensure data confidentiality while maintaining minimum overhead for the resource constrained mobile stations. In this paper, we proposed a security framework based on the concept of hyper-encryption to provide data confidentiality for wireless broadband networks.

Comparison of methods for the proportion of true null hypotheses in microarray studies

  • Kang, Joonsung
    • Communications for Statistical Applications and Methods
    • /
    • v.27 no.1
    • /
    • pp.141-148
    • /
    • 2020
  • We consider estimating the proportion of true null hypotheses in multiple testing problems. A traditional multiple testing rate, family-wise error rate is too conservative and old to control type I error in multiple testing setups; however, false discovery rate (FDR) has received significant attention in many research areas such as GWAS data, FMRI data, and signal processing. Identify differentially expressed genes in microarray studies involves estimating the proportion of true null hypotheses in FDR procedures. However, we need to account for unknown dependence structures among genes in microarray data in order to estimate the proportion of true null hypothesis since the genuine dependence structure of microarray data is unknown. We compare various procedures in simulation data and real microarray data. We consider a hidden Markov model for simulated data with dependency. Cai procedure (2007) and a sliding linear model procedure (2011) have a relatively smaller bias and standard errors, being more proper for estimating the proportion of true null hypotheses in simulated data under various setups. Real data analysis shows that 5 estimation procedures among 9 procedures have almost similar values of the estimated proportion of true null hypotheses in microarray data.

The Developing of Analytical Statistics System for the Efficiency of Defense Management (국방경영 효율화를 위한 분석형 통계시스템 구축)

  • Lee, Jung-Man
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.38 no.3
    • /
    • pp.87-94
    • /
    • 2015
  • Recently, management based on statistical data has become a big issue and the importance of the statistics has been emphasized for the management innovation in the defense area. However, the Military Management based on the statistics is hard to expect because of the shortage of the statistics in the military. There are many military information systems having great many data created in real time. Since the infrastructure for gathering data form the many systems and making statistics by using gathered data is not equipped, the usage of the statistics is poor in the military. The Analytical Defense Statistics System is designed to improve effectively the defense management in this study. The new system having the sub-systems of Data Management, Analysis and Service can gather the operational data from interlocked other Defense Operational Systems and produce Defense Statistics by using the gathered data beside providing statistics services. Additionally, the special function for the user oriented statistics production is added to make new statistics by handling many statistics and data. The Data Warehouse is considered to manage the data and Online Analytical Processing tool is used to enhance the efficiency of the data handling. The main functions of the R, which is a well-known analysis program, are considered for the statistical analysis. The Quality Management Technique is applied to find the fault from the data of the regular and irregular type. The new Statistics System will be the essence of the new technology like as Data Warehouse, Business Intelligence, Data Standardization and Statistics Analysis and will be helpful to improve the efficiency of the Military Management.

Distributed Continuous Query Processing Scheme for RFID Data Stream (RFID 데이터 스트림에 대한 분산 연속질의 처리 기법)

  • Ahn, Sung-Woo;Hong, Bong-Hee;Jung, Dong-Gyu
    • Journal of the Institute of Electronics Engineers of Korea CI
    • /
    • v.46 no.4
    • /
    • pp.1-12
    • /
    • 2009
  • An RFID application needs to collect product's information scattered over the RFID network efficiently according to the globalization of RFID applied enterprises. To be informed of the stock status of products promptly in the supply chain network, especially, it is necessary to support queries that retrieve statistical information of tagged products. Since existing RFID network does not provide these kinds of queries, however, an application should request a query to several RFID middleware systems and analyze collected data directly. This process makes an application do a heavy computation for retrieving statistical information. To solve this problem, we define a new Distributed Continuous Query that finds information of tagged products from the global RFID network and provides statistical information to RFID applications. We also propose a Distributed Continuous Query System to process the distributed continuous query efficiently. To find out the movement of products via multiple RFID systems in real time, our proposed system uses Pedigree which represents trade information of items. Our system can also reduce the cost of query processing for removing duplicated data from multiple middleware systems by using Pedigree.

TV Advertsing Information-Processing Competencies of Children Consumers Based on Consumer Socialization (소비자사회화측면에서 본 아동소비자의 TV광고정보처리능력)

  • 박수경;이기춘
    • Journal of Families and Better Life
    • /
    • v.8 no.1
    • /
    • pp.31-47
    • /
    • 1990
  • The purpose of this study were to investigate: 1) The level of information-processing competencies of children consumers. 2) The differences of information-processing competencises of children consumers according to the cognitive development stage, the variables related to social learning and the socio-ecomonic variables. 3) The independent influences of variables related to information-processing competencies of children consumers. 4) The independent influences of variables related to information-processing competencies of children consumers according to the cognitive development stage. For these purposes, a survey was conducted using questionaires. The questionaires consisted of mother's and child's were distributed to children of second and 6th grade and their mothers of 5 elementary schools in Seoul 634 cases were selected for data analysis. Then, these data were analyzed with statistical methods such as Frequency Distribution, Percentile, Mean, One-way Anova, Scheff -Test, T-Test, Pearson's Correlation and Multiple Regression Analysis. From these finding, the follawing suggestions are made. First, to improve information processing competencies of children consumers, children consumers should be offered the consumer information that is suitable to child's cognitive development stage. Second, the consumer education and learning practical discriminatory competencies on TV ads. should be conducted for children consumers and his mothers. Third, children consumers should have the opportunities to practice as a consumer. The last, children consumers in low-income should be offered the specially educational program.

  • PDF

A Study on the Statistical Status of By-products from Korean Seafood processing for Utilization of Biomaterials (바이오소재 활용을 위한 국내 수산가공부산물의 통계 현황 연구)

  • Soeon, Ahn;Duckhee, Jang;Do-Hyung, Kang
    • Journal of Marine Bioscience and Biotechnology
    • /
    • v.14 no.2
    • /
    • pp.124-132
    • /
    • 2022
  • By-products from fisheries produced in Korea are of the same industrial material as imported raw materials and are valuable resources for marine bioindustries. Securing raw materials for the mass production of functional materials is one of the main objectives for marine bioindustrial development. The use of fishery by-products as raw materials is anticipated to increase rapidly as the biomarket is growing into a promising industry. In this study, data were acquired from an open-source environment to perform exploratory data analysis, and various visualization methods were used to compare fishery production to the production of marine processed products in the year 2020. This study suggested that the amount of seafood processing, types of processing items, and areas where fishery processing residue is generated, should be able to secure hygienic raw material supply in large quantities. Thus far, it has been found that the Gyeonggi-do and Busan province, where HACCP-certified processing facilities are concentrated, and the local government Seafood Cluster and the Smart Aquaculture Cluster are at the forefront of stable, mass production of raw materials.