Journal of Electrical Engineering and Technology

The Korean Institute of Electrical Engineers (대한전기학회)
권호 표지
DOI QR코드

DOI QR Code

Design of Space Search-Optimized Polynomial Neural Networks with the Aid of Ranking Selection and L2-norm Regularization

  • Wang, Dan (School of Computer Science and Information Engineering, Tianjin University of Science &Technology) ;
  • Oh, Sung-Kwun (Dept. of Electrical Engineering, The University of Suwon, Key Laboratory of Complex Systems and Intelligent Computing in Universities of Shandong, Linyi University) ;
  • Kim, Eun-Hu (Dep. of Electrical Engineering, The University of Suwon)
  • Received : 2017.12.12
  • Accepted : 2018.02.13
  • Published : 2018.07.01
Download PDF
⟨ Previous Next ⟩

Abstract

The conventional polynomial neural network (PNN) is a classical flexible neural structure and self-organizing network, however it is not free from the limitation of overfitting problem. In this study, we propose a space search-optimized polynomial neural network (ssPNN) structure to alleviate this problem. Ranking selection is realized by means of ranking selection-based performance index (RS_PI) which is combined with conventional performance index (PI) and coefficients based performance index (CPI) (viz. the sum of squared coefficient). Unlike the conventional PNN, L2-norm regularization method for estimating the polynomial coefficients is also used when designing the ssPNN. Furthermore, space search optimization (SSO) is exploited here to optimize the parameters of ssPNN (viz. the number of input variables, which variables will be selected as input variables, and the type of polynomial). Experimental results show that the proposed ranking selection-based polynomial neural network gives rise to better performance in comparison with the neuron fuzzy models reported in the literatures.

Keywords

References

  1. C. Riziotis and A.V. Vasilakos, "Computational intelligence in photonics technology and optical networks: a survey and future perspectives," Information Sciences, vol. 177, pp. 5292-5315, 2007. https://doi.org/10.1016/j.ins.2007.06.012
  2. W. Huang and L. Ding, "Project-Scheduling problem with random time-dependent activity duration times," IEEE Transactions on Engineering Management, vol. 58, pp. 377-387, 2011. https://doi.org/10.1109/TEM.2010.2063707
  3. B. Wu, S. Han, J. Xiao, X. Hu, and J. Fan, "Error compensation based on BP neural network for airborne laser ranging," International Journal for Light and Electron Optics vol. 127, pp. 4083-4088, 2016. https://doi.org/10.1016/j.ijleo.2016.01.066
  4. W. Huang, J. Wang, and J. Liao. "A granular classifier by means of context-based similarity clustering," Journal of Electrical Engineering & Technology, vol. 11, pp. 1383-1394, 2016. https://doi.org/10.5370/JEET.2016.11.5.1383
  5. S. Yilmaz and Y. Oysal. "Fuzzy wavelet neural network models for prediction and identification of dynamical systems," IEEE Transactions on Neural Networks, vol. 21, pp. 1599-1609, 2010. https://doi.org/10.1109/TNN.2010.2066285
  6. E. Kaslik and L.R. Radulescu. "Dynamics of complex-valued fractional-order neural networks," Neural Networks, vol. 89, pp. 39-49, 2017. https://doi.org/10.1016/j.neunet.2017.02.011
  7. A.G. Ivakhnenko, "Polynomial theory of complex systems," IEEE Trans Systems, Man, and Cybernetics, SMC-vol. 1, pp. 364-378, 1971. https://doi.org/10.1109/TSMC.1971.4308320
  8. K.T. Parveen, B. Sanghamitra, and K.P. Sankar, "Multi-Objective Particel Swarm Optimization with time variant inertia and acceleration coefficients," Information Sciences, vol. 177, pp. 5033-5049, 2007. https://doi.org/10.1016/j.ins.2007.06.018
  9. S. Masato, and G. Mitsuo, "Fuzzy multiple objective optimal system design by hybrid genetic algorithm," Applied Soft Computing, vol. 2, pp. 189-196, 2003. https://doi.org/10.1016/S1568-4946(02)00068-6
  10. M. Delgado, M.P. Ceullar, and M.C. Pegalajar, "Multiobjective Hybrid Optimization and Training of Recurrent Neural Networks," IEEE Trans. Syst., Man cybern. -Part B, vol. 38, pp. 381-403, 2008. https://doi.org/10.1109/TSMCB.2007.912937
  11. S.-K. Oh, W. Pedrycz, and B.-J. Park, "Polynomial Neural Networks architecture: analysis and design," Computers and Electrical Engineering, vol. 29, pp. 703-725, 2003. https://doi.org/10.1016/S0045-7906(02)00045-9
  12. SH Zhou, XW Liu, Q Liu, SQ Wang, CZ Zhu, and JP Yin,"Random Fourier extreme learning machine with l(2,1)-norm regularization," Neuro Computing, vol. 174, pp. 143-153, 2016
  13. W. Huang and S.-K. Oh. "Identification of fuzzy inference systems using multi-objective space search algorithm and information granulation," Journal of Electrical Engineering & Technology, vol. 6, pp. 853-866, 2011. https://doi.org/10.5370/JEET.2011.6.6.853
  14. W. Huang and J. Wang, "Design of polynomial fuzzy radial basis function neural networks based on nonsymmetric fuzzy clustering and parallel optimization," Mathematical Problems in Engineering, vol. 2013, pp. 1-11, 2013
  15. Z. Yong, W. Xiaobei, X. Zongyi, and H. Weili, "On generating interpretable and precise fuzzy systems based on Pareto multi-objective cooperative coevolutionary algorithm," Applied Soft Computing, vol. 11, pp. 1284-1294, 2011 https://doi.org/10.1016/j.asoc.2010.03.005
  16. S.-K. Oh and W. Pedrycz, "Fuzzy polynomial neuronbased self-organizing neural networks," Int. J. of General Systems, vol. 32, pp. 237-250, 2003. https://doi.org/10.1080/0308107031000090756
  17. J.-N. Choi, S.-K. Oh, and W. Pedrycz, "Identification of fuzzy models using a successive tuning method with a variant identification ratio," Fuzzy Sets and Systems, vol. 159, pp. 2873-2889, 2008 https://doi.org/10.1016/j.fss.2007.12.031
  18. R. Alcala, P. Ducange, F. Herrera, B. Lazzerini, and Marcelloni F, "A multiobjective evolutionary approach to concurrently learn rule and data bases of linguistic fuzzy-rule-based systems," IEEE Transactions on Fuzzy Systems, vol. 17, pp. 1106-1122, 2009. https://doi.org/10.1109/TFUZZ.2009.2023113
  19. R. Alcala, M. J. Gacto, and F. Herrera," A fast and scalable multiobjective genetic fuzzy System for linguistic fuzzy modeling in high-dimensional regression problems," IEEE Transactions on Fuzzy Systems, vol. 19, pp. 666-681, 2011. https://doi.org/10.1109/TFUZZ.2011.2131657
  20. W. Pedrycz, and K.-C. Kwak, "The Development of Incremental Models," IEEE Trans. Fuzzy Systems, vol. 15, pp. 507-518, 2007. https://doi.org/10.1109/TFUZZ.2006.889967
  21. W. Pedrycz, and K.-C. Kwak, "Boosting of granular models," Fuzzy Sets and Systems, vol. 157, pp. 2943- 2953, 2006.
  22. J.S.R. Jang and C.T. Sun, "Functional equivalence between radial basis function networks and fuzzy inference systems," IEEE Trans. Neural Networks, vol. 4, pp. 156-158, 1993 https://doi.org/10.1109/72.182710
  23. W. Pedrycz, and K.-C. Kwak, "Linguistic models as a framework of user-centric system modeling," IEEE Trans. SMC-A, vol. 36, pp. 727-745, 2006
  24. M. Antonelli, P. Ducange, and F. Marcelloni, "An efficient multi-objective evolutionary fuzzy system for regression problems," International Journal of Approximate Reasoning, vol. 54, pp. 1434-1451, 2013. https://doi.org/10.1016/j.ijar.2013.06.005
  25. J. Liu, F. Chung, and S. Wang, "Bayesian zero-order TSK fuzzy system modeling," Applied Soft Computing, vol. 55, pp. 253-264, 2017. https://doi.org/10.1016/j.asoc.2017.01.040
  26. W. Huang, S.-K. Oh, and W. Pedrycz, "Fuzzy wavelet polynomial neural networks: analysis and design," IEEE Trans. Fuzzy Systems, vol. 25, no. 5, pp. 1329- 1341, 2017. https://doi.org/10.1109/TFUZZ.2016.2612267