• 제목/요약/키워드: Data filtering

검색결과 1,777건 처리시간 0.028초

머신러닝 기반의 안전도 데이터 필터링 모델 (Electrooculography Filtering Model Based on Machine Learning)

  • 홍기현;이병문
    • 한국멀티미디어학회논문지
    • /
    • 제24권2호
    • /
    • pp.274-284
    • /
    • 2021
  • Customized services to a sleep induction for better sleepcare are more effective because of different satisfaction levels to users. The EOG data measured at the frontal lobe when a person blinks his eyes can be used as biometric data because it has different values for each person. The accuracy of measurement is degraded by a noise source, such as toss and turn. Therefore, it is necessary to analyze the noisy data and remove them from normal EOG by filtering. There are low-pass filtering and high-pass filtering as filtering using a frequency band. However, since filtering within a frequency band range is also required for more effective performance, we propose a machine learning model for the filtering of EOG data in this paper as the second filtering method. In addition, optimal values of parameters such as the depth of the hidden layer, the number of nodes of the hidden layer, the activation function, and the dropout were found through experiments, to improve the performance of the machine learning filtering model, and the filtering performance of 95.7% was obtained. Eventually, it is expected that it can be used for effective user identification services by using filtering model for EOG data.

센서태그 통합 데이터 필터링에 관한 연구 (Cooperative Data Stream Filtering for Sensor Tag)

  • 류승완;오슬기;박세권;오동옥
    • 한국통신학회논문지
    • /
    • 제36권8A호
    • /
    • pp.683-690
    • /
    • 2011
  • 센서 태그의 데이터는 태그 정보와 센싱 정보를 동시에 가지며 미들웨어 또는 상위 레벨에서의 필터링 및 가공이 필요하다는 특정을 가지고 있다. 기존의 필터링 알고리즘에서는 태그데이터와 센서 데이터를 각각 필터링하는 알고리즘이 주로 제안되었다. 그러나 센서 태그의 사용 요구는 점차 증가하고 있으며, 사용요구에 적합한 필터링을 위해서는 센싱 데이터와 RFID 데이터를 통합 처리할 수 있는 새로운 필터링 알고리즘이 필요하다. 본 논문에서 제안하는 필터링 알고리즘에서는 각 태그의 시간 축에 대한 필터링만을 고려하는 것이 아니라 공간적으로 근접한 태그의 데이터도 함께 고려하여 필터링하여 오류 및 이벤트 검출의 정확성을 향상시키고 데이터의 대표값 저장으로 데이터 저장에 필요한 비용을 감소시킬 수 있다.

다중선형 회귀분석에 의한 LiDAR 자료의 필터링 자동화 기법 (An Filtering Automatic Technique of LiDAR Data by Multiple Linear Regression Analysis)

  • 최승필;조지현;김준성
    • 대한공간정보학회지
    • /
    • 제19권4호
    • /
    • pp.109-118
    • /
    • 2011
  • 본 연구는 지면 데이터�V을 이용하여 다중선형 회귀분석에 의한 평면방정식을 도출하여 전역필터링 한 것을 기준으로 전체 데이터�V을 이용하여 도출된 평면방정식으로 전역필터링 한 것과 가상격자별로 평면방정식을 도출하여 지역필터링을 수행한 결과를 분석하여 정확도를 평가하였다. 그 결과 지면 데이터�V을 이용한 전역필터링의 평균정확도를 기준으로 전체 데이터�V을 이용한 전역필터링의 정확도는 약 2~3%정도 떨어지고, 가상격자를 이용한 지역필터링의 정확도는 약 2~4% 떨어지는 것으로 나타났다. 특히 가상격자가 3~4cm일 때 기준자료와 약 2%의 정확도의 차이가 나타낸 것으로 보아 가상격자 사이즈를 라이다 스캔간격의 3~4배 크기로 지정하여 필터링 하는 것이 바람직 할 것으로 판단된다. 따라서 필터링의 적용방법에 따라 평균정확도가 차이가 발생하였으며, 향후 보다 다양한 실제지형을 선정하여 필터링의 정확도에 대한 연구가 필요할 것으로 생각된다.

Improvement of Collaborative Filtering Algorithm Using Imputation Methods

  • Jeong, Hyeong-Chul;Kwak, Min-Jung;Noh, Hyun-Ju
    • Journal of the Korean Data and Information Science Society
    • /
    • 제14권3호
    • /
    • pp.441-450
    • /
    • 2003
  • Collaborative filtering is one of the most widely used methodologies for recommendation system. Collaborative filtering is based on a data matrix of each customer's preferences and frequently, there exits missing data problem. We introduced two imputation approach (multiple imputation via Markov Chain Monte Carlo method and multiple imputation via bootstrap method) to improve the prediction performance of collaborative filtering and evaluated the performance using EachMovie data.

  • PDF

Data Sparsity and Performance in Collaborative Filtering-based Recommendation

  • Kim Jong-Woo;Lee Hong-Joo
    • Management Science and Financial Engineering
    • /
    • 제11권3호
    • /
    • pp.19-45
    • /
    • 2005
  • Collaborative filtering is one of the most common methods that e-commerce sites and Internet information services use to personalize recommendations. Collaborative filtering has the advantage of being able to use even sparse evaluation data to predict preference scores for new products. To date, however, no in-depth investigation has been conducted on how the data sparsity effect in customers' evaluation data affects collaborative filtering-based recommendation performance. In this study, we analyzed the sparsity effect and used a hybrid method based on customers' evaluations and purchases collected from an online bookstore. Results indicated that recommendation performance decreased monotonically as sparsity increased, and that performance was more sensitive to sparsity in evaluation data rather than in purchase data. Results also indicated that the hybrid use of two different types of data (customers' evaluations and purchases) helped to improve the recommendation performance when evaluation data were highly sparse.

센서 네트워크에서 계층적 필터링을 이용한 에너지 절약 방안 (An Energy Saving Method using Hierarchical Filtering in Sensor Networks)

  • 김진수
    • 한국산학기술학회논문지
    • /
    • 제8권4호
    • /
    • pp.768-774
    • /
    • 2007
  • 본 논문에서는 센서 네트워크의 수명을 길게 하기 위해 각 센서 및 클러스터 헤드에서의 데이터 전송량을 줄이기 위한 방법을 제안한다. 즉, 센서의 에너지 소모를 줄이기 위해 계층적 필터링을 제안한다. 계층적 필터링이란 센서 네트워크를 두 계층으로 나누어 필터링하는 것이다. 1계층 필터링은 클러스터 멤버에서 클러스터 헤드로 데이터를 전송시 필터링을 수행하고, 2계층 필터링은 클러스터 헤드에서 기지국으로 데이터를 전송시 필터링을 수행한다. 이는 일반적으로 필터의 폭을 넓혀 필터링을 많이 하는 것보다 필터링 효율은 증대시키면서 필터링에 따른 데이터 부정확성을 최소한 줄이는 효과를 가진다.

  • PDF

A Study on Filtering Techniques for Dynamic Analysis of Data Races in Multi-threaded Programs

  • Ha, Ok-Kyoon;Yoo, Hongseok
    • 한국컴퓨터정보학회논문지
    • /
    • 제22권11호
    • /
    • pp.1-7
    • /
    • 2017
  • In this paper, we introduce three monitoring filtering techniques which reduce the overheads of dynamic data race detection. It is well known that detecting data races dynamically in multi-threaded programs is quite hard and troublesome task, because the dynamic detection techniques need to monitor all execution of a multi-threaded program and to analyse every conflicting memory and thread operations in the program. Thus, the main drawback of the dynamic analysis for detecting data races is the heavy additional time and space overheads for running the program. For the practicality, we also empirically compare the efficiency of three monitoring filtering techniques. The results using OpenMP benchmarks show that the filtering techniques are practical for dynamic data race detection, since they reduce the average runtime overhead to under 10% of that of the pure detection.

A Model-based Collaborative Filtering Through Regularized Discriminant Analysis Using Market Basket Data

  • Lee, Jong-Seok;Jun, Chi-Hyuck;Lee, Jae-Wook;Kim, Soo-Young
    • Management Science and Financial Engineering
    • /
    • 제12권2호
    • /
    • pp.71-85
    • /
    • 2006
  • Collaborative filtering, among other recommender systems, has been known as the most successful recommendation technique. However, it requires the user-item rating data, which may not be easily available. As an alternative, some collaborative filtering algorithms have been developed recently by utilizing the market basket data in the form of the binary user-item matrix. Viewing the recommendation scheme as a two-class classification problem, we proposed a new collaborative filtering scheme using a regularized discriminant analysis applied to the binary user-item data. The proposed discriminant model was built in terms of the major principal components and was used for predicting the probability of purchasing a particular item by an active user. The proposed scheme was illustrated with two modified real data sets and its performance was compared with the existing user-based approach in terms of the recommendation precision.

Model of dynamic clustering-based energy-efficient data filtering for mobile RFID networks

  • Vo, Viet Minh Nhat;Le, Van Hoa
    • ETRI Journal
    • /
    • 제43권3호
    • /
    • pp.427-435
    • /
    • 2021
  • Data filtering is an essential task for improving the energy efficiency of radiofrequency identification (RFID) networks. Among various energy-efficient approaches, clustering-based data filtering is considered to be the most effective solution because data from cluster members can be filtered at cluster heads before being sent to base stations. However, this approach quickly depletes the energy of cluster heads. Furthermore, most previous studies have assumed that readers are fixed and interrogate mobile tags in a workspace. However, there are several applications in which readers are mobile and interrogate fixed tags in a specific area. This article proposes a model for dynamic clustering-based data filtering (DCDF) in mobile RFID networks, where mobile readers are re-clustered periodically and the cluster head role is rotated among the members of each cluster. Simulation results show that DCDF is effective in terms of balancing energy consumption among readers and prolonging the lifetime of the mobile RFID networks.

여과기법 보안효율을 높이기 위한 센서네트워크 클러스터링 방법 (Enhancing Method to make Cluster for Filtering-based Sensor Networks)

  • 김병희;조대호
    • 한국정보통신설비학회:학술대회논문집
    • /
    • 한국정보통신설비학회 2008년도 정보통신설비 학술대회
    • /
    • pp.141-145
    • /
    • 2008
  • Wireless sensor network (WSN) is expected to be used in many applications. However, sensor nodes still have some secure problems to use them in the real applications. They are typically deployed on open, wide, and unattended environments. An adversary using these features can easily compromise the deployed sensor nodes and use compromised sensor nodes to inject fabricated data to the sensor network (false data injection attack). The injected fabricated data drains much energy of them and causes a false alarm. To detect and drop the injected fabricated data, a filtering-based security method and adaptive methods are proposed. The number of different partitions is important to make event report since they can make a correctness event report if the representative node does not receive message authentication codes made by the different partition keys. The proposed methods cannot guarantee the detection power since they do not consider the filtering scheme. We proposed clustering method for filtering-based secure methods. Our proposed method uses fuzzy system to enhance the detection power of a cluster.

  • PDF