Journal of Information Processing Systems

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

A Detailed Analysis of Classifier Ensembles for Intrusion Detection in Wireless Network

  • Tama, Bayu Adhi (Laboratory of Information Security and Internet Applications, Pukyong National University) ;
  • Rhee, Kyung-Hyune (Faculty of Computer Science, Sriwijaya University)
  • Received : 2016.02.26
  • Accepted : 2016.07.05
  • Published : 2017.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

Intrusion detection systems (IDSs) are crucial in this overwhelming increase of attacks on the computing infrastructure. It intelligently detects malicious and predicts future attack patterns based on the classification analysis using machine learning and data mining techniques. This paper is devoted to thoroughly evaluate classifier ensembles for IDSs in IEEE 802.11 wireless network. Two ensemble techniques, i.e. voting and stacking are employed to combine the three base classifiers, i.e. decision tree (DT), random forest (RF), and support vector machine (SVM). We use area under ROC curve (AUC) value as a performance metric. Finally, we conduct two statistical significance tests to evaluate the performance differences among classifiers.

Keywords

References

  1. B. A. Tama and K. H. Rhee, "Performance analysis of multiple classifier system in DoS attack detection," in Information Security Applications, LNCS, vol. 9503. Cham, Switzerland: Springer International Publishing, 2016, pp. 339-347.
  2. B. A. Tama and K. H. Rhee, "A combination of PSO-based feature selection and tree-based classifiers ensemble for intrusion detection systems," in Advances in Computer Science and Ubiquitous Computing. Singapore: Springer Singapore, 2015, pp. 489-495.
  3. B. A. Tama and K. H. Rhee, "Data mining techniques in DoS/DDoS attack detection: a literature review," Information, vol. 18, no. 8, pp. 3739-3747, 2015.
  4. M. P. Ponti, "Combining classifiers: from the creation of ensembles to the decision fusion," in Proceedings of the 24th SIBGRAPI Conference on Graphics, Patterns and Images Tutorials (SIBGRAPI-T), Alagoas, Brazil, 2011, pp. 1-10.
  5. D. Ruta and B. Gabrys, "Classifier selection for majority voting," Information Fusion, vol. 6, no. 1, pp. 63-81, 2005. https://doi.org/10.1016/j.inffus.2004.04.008
  6. A. K. Seewald, "How to make stacking better and faster while also taking care of an unknown weakness," in Proceedings of the 19th International Conference on Machine Learning, Nevada, LA, 2002, pp. 554-561.
  7. J. R. Quinlan, C4.5: Programs for Machine Learning. San Mateo, CA: Morgan Kaufmann, 1993.
  8. L. Breiman, "Random forests," Machine Learning, vol. 45, no. 1, pp. 5-32, 2001. https://doi.org/10.1023/A:1010933404324
  9. V. N. Vapnik, Statistical Learning Theory. New York, NY: Wiley, 1998.
  10. D. W. F. V. Vilela, E. T. Ferreira, A. A. Shinoda, N. V. de Souza Araujo, R. de Oliveira, and V. E. Nascimento, "A dataset for evaluating intrusion detection systems in IEEE 802.11 wireless networks," in Proceedings of the IEEE Colombian Conference on Communications and Computing (COLCOM), Bogota, Colombia, 2014, pp. 1-5.
  11. P. Garcia-Teodoro, J. Diaz-Verdejo, G. Macia-Fernandez, and E. Vazquez, "Anomaly-based network intrusion detection: techniques, systems and challenges," Computers & Security, vol. 28, no. 1-2, pp. 18-28, 2009. https://doi.org/10.1016/j.cose.2008.08.003
  12. C. F. Tsai, Y. F. Hsu, C. Y. Lin, and W. Y. Lin, "Intrusion detection by machine learning: a review," Expert Systems with Applications, vol. 36, no. 10, pp. 11994-12000, 2009. https://doi.org/10.1016/j.eswa.2009.05.029
  13. N. C. Oza and K. Tumer, "Classifier ensembles: select real-world applications," Information Fusion, vol. 9, no. 1, pp. 4-20, 2008. https://doi.org/10.1016/j.inffus.2007.07.002
  14. L. Rokach, "Ensemble-based classifiers," Artificial Intelligence Review, vol. 33, no. 1, pp. 1-39, 2010. https://doi.org/10.1007/s10462-009-9124-7
  15. L. I. Kuncheva, Combining Pattern Classifiers: Methods and Algorithm, 2nd ed. Hoboken, NJ: John Wiley & Sons, 2014.
  16. M. P. Sesmero, A. I. Ledezma, and A. Sanchis, "Generating ensembles of heterogeneous classifiers using stacked generalization," Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, vol. 5, no. 1, pp. 21-34, 2015. https://doi.org/10.1002/widm.1143
  17. K. M. Ting and I. H. Witten, "Issues in stacked generalization," Journal of Artificial Intelligence Research, vol. 10, pp. 271-289, 1999. https://doi.org/10.1613/jair.594
  18. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification. New York, NY: John Wiley & Sons, 2001.
  19. J. R. Quinlan, "Induction of decision trees," Machine Learning, vol. 1, no. 1, pp. 81-106, 1986. https://doi.org/10.1007/BF00116251
  20. P. Cichosz, Data Mining Algorithms: Explained Using R. Chichester, UK: John Wiley & Sons, 2015.
  21. C. Cortes and V. Vapnik, "Support-vector networks," Machine Learning, vol. 20, no. 3, pp. 273-297, 1995. https://doi.org/10.1007/BF00994018
  22. C. W. Hsu, C. C. Chang, and C. J. Lin, A Practical Guide to Support Vector Classification. Taipei City, Taiwan: National Taiwan University, 2010.
  23. J. Demsar, "Statistical comparisons of classifiers over multiple data sets," Journal of Machine Learning Research, vol. 7, pp. 1-30, 2006.
  24. T. Fawcett, "An introduction to ROC analysis," Pattern Recognition Letters, vol. 27, no. 8, pp. 861-874, 2006. https://doi.org/10.1016/j.patrec.2005.10.010
  25. F. Provost, T. Fawcett, and R. Kohavi, "The case against accuracy estimation for comparing induction algorithms," in Proceedings of the 15th International Conference on Machine Learning (ICML-98), Madison, WI, 1998, pp. 445-453.
  26. M. Friedman, "A comparison of alternative tests of significance for the problem of m rankings," The Annals of Mathematical Statistics, vol. 11, no. 1, pp. 86-92, 1940. https://doi.org/10.1214/aoms/1177731944
  27. N. Japkowicz and M. Shah, Evaluating Learning Algorithms: A Classification Perspective. New York, NY: Cambridge University Press, 2011.
  28. S. Chebrolu, A. Abraham, and J. P. Thomas, "Feature deduction and ensemble design of intrusion detection systems," Computers & Security, vol. 24, no. 4, pp. 295-307, 2005. https://doi.org/10.1016/j.cose.2004.09.008
  29. S. Mukkamala, A. H. Sung, and A. Abraham, "Intrusion detection using an ensemble of intelligent paradigms," Journal of Network and Computer Applications, vol. 28, no. 2, pp. 167-182, 2005. https://doi.org/10.1016/j.jnca.2004.01.003
  30. S. Peddabachigari, A. Abraham, C. Grosan, and J. Thomas, "Modeling intrusion detection system using hybrid intelligent systems," Journal of Network and Computer Applications, vol. 30, no. 1, pp. 114-132, 2007. https://doi.org/10.1016/j.jnca.2005.06.003
  31. W. Hu, W. Hu, and S. Maybank, "AdaBoost-based algorithm for network intrusion detection," IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 38, no. 2, pp. 577-583, 2008. https://doi.org/10.1109/TSMCB.2007.914695
  32. J. B. D. Cabrera, C. Gutierrez, and R. K. Mehra, "Ensemble methods for anomaly detection and distributed intrusion detection in mobile ad-hoc networks," Information Fusion, vol. 9, no. 1, pp. 96-119, 2008. https://doi.org/10.1016/j.inffus.2007.03.001
  33. G. Giacinto, R. Perdisci, M. Del Rio, and F. Roli, "Intrusion detection in computer networks by a modular ensemble of one-class classifiers," Information Fusion, vol. 9, no. 1, pp. 69-82, 2008. https://doi.org/10.1016/j.inffus.2006.10.002
  34. S. S. S. Sindhu, S. Geetha, and A. Kannan, "Decision tree based light weight intrusion detection using a wrapper approach," Expert Systems with Applications, vol. 39, no. 1, pp. 129-141, 2012. https://doi.org/10.1016/j.eswa.2011.06.013
  35. M. Govindarajan and R. M. Chandrasekaran, "Intrusion detection using neural based hybrid classification methods," Computer Networks, vol. 55, no. 8, pp. 1662-1671, 2011. https://doi.org/10.1016/j.comnet.2010.12.008