DOI QR코드

DOI QR Code

Machine Scheduling Models Based on Reinforcement Learning for Minimizing Due Date Violation and Setup Change

납기 위반 및 셋업 최소화를 위한 강화학습 기반의 설비 일정계획 모델

  • Yoo, Woosik (Department of Industrial and Management Engineering, Incheon National University) ;
  • Seo, Juhyeok (Department of Industrial and Management Engineering, Incheon National University) ;
  • Kim, Dahee (Department of Industrial and Management Engineering, Incheon National University) ;
  • Kim, Kwanho (Department of Industrial and Management Engineering, Incheon National University)
  • Received : 2019.04.10
  • Accepted : 2019.08.16
  • Published : 2019.08.31

Abstract

Recently, manufacturers have been struggling to efficiently use production equipment as their production methods become more sophisticated and complex. Typical factors hindering the efficiency of the manufacturing process include setup cost due to job change. Especially, in the process of using expensive production equipment such as semiconductor / LCD process, efficient use of equipment is very important. Balancing the tradeoff between meeting the deadline and minimizing setup cost incurred by changes of work type is crucial planning task. In this study, we developed a scheduling model to achieve the goal of minimizing the duedate and setup costs by using reinforcement learning in parallel machines with duedate and work preparation costs. The proposed model is a Deep Q-Network (DQN) scheduling model and is a reinforcement learning-based model. To validate the effectiveness of our proposed model, we compared it against the heuristic model and DNN(deep neural network) based model. It was confirmed that our proposed DQN method causes less due date violation and setup costs than the benchmark methods.

최근 제조업체들은 제품의 생산방식이 고도화 되고, 복잡해지면서 생산 장비를 효율적으로 사용하는데 어려움을 겪고 있다. 제조공정의 효율성을 방해하는 대표적인 요인들로는 작업물 종류 변경(job change)으로 인한 작업 준비 비용(Setup Cost) 등이 있다. 특히 반도체/LCD 공정과 같이 고가의 생산 장비를 사용하는 공정의 경우 장비의 효율적인 사용이 매우 중요한데, 상호 충돌하는 의사결정인 납기 준수를 최대화 하는 것과 작업물 종류 변경으로 인한 작업 준비 비용을 최소화 하는 것 사이에서 균형을 유지하는 것은 매우 어려운 일이다. 본 연구에서는 납기와 작업 준비 비용이 있는 병렬기계에서 강화학습을 활용하여 납기 및 셋업 비용의 최소화 목표를 달성하는 일정계획 모델을 개발하였다. 제안하는 모델은 DQN(Deep Q-Network) 일정계획 모델로 강화학습기반의 모델이다. 제안모델의 효율성을 측정하기 위해 DQN 모델과 기존에 개발하였던 심층 신경망 기반의 일정계획 생성기법과 휴리스틱 원칙의 결과를 비교하였다. 비교 결과 DQN 일정계획 생성기법이 심층신경망 방식과 휴리스틱 원칙에 비하여 납기 및 셋업 비용이 적은 것을 확인할 수 있었다.

Keywords

References

  1. Cho, Y. C., Cho, H. C., Kim, J. W., Zhang, L., and Jeon, H. T., “A Study on Optimal Scheduling of Multi-Spinner's Manufacturing Process Using Artificial Neural Network,” International Journal of Fuzzy Logic and Intelligent Systems, Vol. 18, No. 1, pp. 157-160, 2008.
  2. Demirkov, E. and Uzsoy, R., “Decomposition methods for reentrant flow shops with sequence-dependent setup times,” Journal of Scheduling, Vol. 3, No. 3, pp. 155-177, 2010. https://doi.org/10.1002/(SICI)1099-1425(200005/06)3:3<155::AID-JOS39>3.0.CO;2-E
  3. Gabel, T. and Riedmiller, M., “Distributed policy search reinforcement learning for job-shop scheduling tasks,” International Journal of Production Research, Vol. 50, No. 1, pp. 41-61, 2012. https://doi.org/10.1080/00207543.2011.571443
  4. Jeong, H. S., Kim, M. W., Lee, B. J., Kim, K. T., and Youn, H. Y., "An Neural Network Approach to Job-shop Scheduling based on Reinforcement Learning," Proceedings of the Korean Society of Computer Information Conference, Vol. 26, No. 2, pp. 47-48, 2018.
  5. Kang, J., “Feasibility Study on the Reinforcement Learning for a Dynamic Scheduling Problem,” Journal of advanced engineering and technology, Vol. 10, No. 4, pp. 453-456, 2017. https://doi.org/10.35272/jaet.2017.10.4.453
  6. Kang, Y. H., Lee, Y. S., and Shin, H. J., “Problem space based search algorithm for manufacturing process with rework probabilities affecting product quality and tardiness,” Journal of the Korea Academia-Industrial Cooperation Society, Vol. 10, No. 7, pp. 1702-1710, 2009. https://doi.org/10.5762/KAIS.2009.10.7.1702
  7. Kim, A. K., "Ensemble-based Quality Classification and Deep Reinforcement Learning-based Production Scheduling: Ensemble-based Quality Classification and Deep Reinforcement Learning-based Production Scheduling," Ph.D. dissertation, KyungHee University, 2018.
  8. Kim, H. S., Park, D. J., and Jeong, I. J., "Railway Capacity Allocation Modeling using Genetic Algorithm," Transportation Research Record: Journal of the Transportation Research Board, Vol. 2608, pp. 1223-1230, 2016.
  9. Kim, J. Y., "Packet scheduling algorithm using deep Q-Network in wireless network," M.S dissertation, Yonsei University, 2018.
  10. Lee, D. H. and Kim, K. H., “Deep Learning Based Prediction Method of Long-term Photovoltaic Power Generation Using Meteorological and Seasonal Information,” The Journal of Society for e-Business Studies, Vol. 24, No. 1, pp. 1-16, 2019.
  11. Lee, Y. S., "A study on Developing algorithms for Manufacturing networks using meta-Heuristics," Ph.D. Thesis, Sangmyung University, 2000.
  12. Mazdeh, M. M., Nakhjavani, A. K., and Zareei, A., “Minimizing Total Weighted Tardiness with Drop Dead Dates in Single Machine Scheduling Problem,” International Journal of Industrial Engineering & Production Research, Vol. 21, No. 2, pp. 89-95, 2010.
  13. Oh, S. W., Lee, H. K., Shin, J. Y., and Lee, J. H., “Antibiotics-Resistant Bacteria Infection Prediction Based on Deep Learning,” The Journal of Society for e-Business Studies, Vol. 24, No. 1, pp. 105-120, 2019.
  14. Ryu, J. W., Kang, M. K., and Kim, M. W., “A Clustering Algorithm based on Heuristic Evolution Algorithm,” Korean Institute of Information Scientists and Engineers, Vol. 27, No. 2, pp. 78-80, 2000.
  15. Shahrabia, J., Adibib, M. A., and Mahootchia, M., "A reinforcement learning approach to parameter estimation in dynamic job shop scheduling," Computers and Industrial Engineering, Vol. 110, pp. 75-82, 2016. https://doi.org/10.1016/j.cie.2017.05.026