• Title/Summary/Keyword: datasets

Search Result 2,091, Processing Time 0.03 seconds

An Improved Automated Spectral Clustering Algorithm

  • Xiaodan Lv
    • Journal of Information Processing Systems
    • /
    • v.20 no.2
    • /
    • pp.185-199
    • /
    • 2024
  • In this paper, an improved automated spectral clustering (IASC) algorithm is proposed to address the limitations of the traditional spectral clustering (TSC) algorithm, particularly its inability to automatically determine the number of clusters. Firstly, a cluster number evaluation factor based on the optimal clustering principle is proposed. By iterating through different k values, the value corresponding to the largest evaluation factor was selected as the first-rank number of clusters. Secondly, the IASC algorithm adopts a density-sensitive distance to measure the similarity between the sample points. This rendered a high similarity to the data distributed in the same high-density area. Thirdly, to improve clustering accuracy, the IASC algorithm uses the cosine angle classification method instead of K-means to classify the eigenvectors. Six algorithms-K-means, fuzzy C-means, TSC, EIGENGAP, DBSCAN, and density peak-were compared with the proposed algorithm on six datasets. The results show that the IASC algorithm not only automatically determines the number of clusters but also obtains better clustering accuracy on both synthetic and UCI datasets.

Deciphering Key Genes of Proliferative and Secretory Phase Using Integrated Transcriptomics and Network Analysis

  • Payal Gupta;Shriya Dube;Payal Priyadarshini;Shanvi Singh;Anasuya Pravallika R;Vijay Lakshmi Srivastava;Abhishek Sengupta;Priyanka Narad
    • Microbiology and Biotechnology Letters
    • /
    • v.51 no.3
    • /
    • pp.317-324
    • /
    • 2023
  • Endometrium receptivity is a complex mechanism of intricate pathways that lead to the shift from the proliferative to the secretory phase. Our goal was to identify high-ranking differentially expressed genes and study the pathways associated with the phenomenon. Raw data were retrieved from six GEO datasets and 705 DEGs were identified through robust ranking aggregation after the integration of five datasets. 20 key genes were identified that were further re-validated in an additional dataset. Supporting evidence through the experimental references confirms them as major biomarkers of the shift from the proliferative to the secretory phase.

Misinformation Detection and Rectification Based on QA System and Text Similarity with COVID-19

  • Insup Lim;Namjae Cho
    • Journal of Information Technology Applications and Management
    • /
    • v.28 no.5
    • /
    • pp.41-50
    • /
    • 2021
  • As COVID-19 spread widely, and rapidly, the number of misinformation is also increasing, which WHO has referred to this phenomenon as "Infodemic". The purpose of this research is to develop detection and rectification of COVID-19 misinformation based on Open-domain QA system and text similarity. 9 testing conditions were used in this model. For open-domain QA system, 6 conditions were applied using three different types of dataset types, scientific, social media, and news, both datasets, and two different methods of choosing the answer, choosing the top answer generated from the QA system and voting from the top three answers generated from QA system. The other 3 conditions were the Closed-Domain QA system with different dataset types. The best results from the testing model were 76% using all datasets with voting from the top 3 answers outperforming by 16% from the closed-domain model.

Enhancing Malware Detection with TabNetClassifier: A SMOTE-based Approach

  • Rahimov Faridun;Eul Gyu Im
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2024.05a
    • /
    • pp.294-297
    • /
    • 2024
  • Malware detection has become increasingly critical with the proliferation of end devices. To improve detection rates and efficiency, the research focus in malware detection has shifted towards leveraging machine learning and deep learning approaches. This shift is particularly relevant in the context of the widespread adoption of end devices, including smartphones, Internet of Things devices, and personal computers. Machine learning techniques are employed to train models on extensive datasets and evaluate various features, while deep learning algorithms have been extensively utilized to achieve these objectives. In this research, we introduce TabNet, a novel architecture designed for deep learning with tabular data, specifically tailored for enhancing malware detection techniques. Furthermore, the Synthetic Minority Over-Sampling Technique is utilized in this work to counteract the challenges posed by imbalanced datasets in machine learning. SMOTE efficiently balances class distributions, thereby improving model performance and classification accuracy. Our study demonstrates that SMOTE can effectively neutralize class imbalance bias, resulting in more dependable and precise machine learning models.

Network Intrusion Detection Using Transformer and BiGRU-DNN in Edge Computing

  • Huijuan Sun
    • Journal of Information Processing Systems
    • /
    • v.20 no.4
    • /
    • pp.458-476
    • /
    • 2024
  • To address the issue of class imbalance in network traffic data, which affects the network intrusion detection performance, a combined framework using transformers is proposed. First, Tomek Links, SMOTE, and WGAN are used to preprocess the data to solve the class-imbalance problem. Second, the transformer is used to encode traffic data to extract the correlation between network traffic. Finally, a hybrid deep learning network model combining a bidirectional gated current unit and deep neural network is proposed, which is used to extract long-dependence features. A DNN is used to extract deep level features, and softmax is used to complete classification. Experiments were conducted on the NSLKDD, UNSWNB15, and CICIDS2017 datasets, and the detection accuracy rates of the proposed model were 99.72%, 84.86%, and 99.89% on three datasets, respectively. Compared with other relatively new deep-learning network models, it effectively improved the intrusion detection performance, thereby improving the communication security of network data.

Simulator-Driven Sieving Data Generation for Aggregate Image Analysis

  • DaeHan Ahn
    • Journal of information and communication convergence engineering
    • /
    • v.22 no.3
    • /
    • pp.249-255
    • /
    • 2024
  • Advancements in deep learning have enhanced vision-based aggregate analysis. However, further development and studies have encountered challenges, particularly in acquiring large-scale datasets. Data collection is costly and time-consuming, posing a significant challenge in acquiring large datasets required for training neural networks. To address this issue, this study introduces a simulation that efficiently generates the necessary data and labels for training neural networks. We utilized a genetic algorithm (GA) to create optimized lists of aggregates based on the specified values of weight and particle size distribution for the aggregate sample. This enabled sample data collection without conducting sieving tests. Our evaluation of the proposed simulation and GA methodology revealed errors of 1.3% and 2.7 g for aggregate size distribution and weight, respectively. Furthermore, we assessed a segmentation model trained with data from the simulation, achieving a promising preliminary F1 score of 78.18 on the actual aggregate image.