Journal of Korea Game Society (한국게임학회 논문지)

Korea Game Society (한국게임학회)
권호 표지
DOI QR코드

DOI QR Code

Generation of AI Agent in Imperfect Information Card Games Using MCTS Algorithm: Focused on Hearthstone

MCTS 기법을 활용한 불완전 정보 카드 게임에서의 인공지능 에이전트 생성 : 하스스톤을 중심으로

  • Oh, Pyeong (Graduate School, Dept. of Interaction Design, Hallym University) ;
  • Kim, Ji-Min (Graduate School, Dept. of Interaction Design, Hallym University) ;
  • Kim, Sun-Jeong (Graduate School, Dept. of Interaction Design, Hallym University) ;
  • Hong, Seokmin (Dept. of Advertising and Public Relations, Hallym University)
  • 오평 (한림대학교 인터랙션디자인) ;
  • 김지민 (한림대학교 인터랙션디자인) ;
  • 김선정 (한림대학교 인터랙션디자인) ;
  • 홍석민 (한림대학교 광고홍보학과)
  • Received : 2016.09.02
  • Accepted : 2016.12.01
  • Published : 2016.12.20
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, many researchers have paid attention to the improved generation of AI agent in the area of game industry. Monte-Carlo Tree Search(MCTS) is one of the algorithms to search an optimal solution through random search with perfect information, and it is suitable for the purpose of calculating an approximate value to the solution of an equation which cannot be expressed explicitly. Games in Trading Card Game(TCG) genre such as the heartstone has imperfect information because the cards and play of an opponent are not predictable. In this study, MCTS is suggested in imperfect information card games so as to generate AI agents. In addition, the practicality of MCTS algorithm is verified by applying to heartstone game which is currently used.

최근 게임분야에서 수준 높은 인공지능 에이전트의 구현은 많은 주목을 받고 있다. 그 중 Monte-Carlo Tree Search(MCTS)는 완전 정보를 가진 게임에서 무작위 탐색을 통해 최적의 해를 구할 수 있는 알고리즘으로, 수식으로 표현되지 않는 경우에 근사치를 계산하는 용도로 적합하다. 하스스톤과 같은 Trading Card Game(TCG) 장르의 게임은 상대방의 카드와 플레이를 예측할 수 없기 때문에 불완전 정보를 가지고 있다. 본 논문에서는 불완전 정보 카드 게임에서 인공지능 에이전트를 생성하기 위해 MCTS 알고리즘을 응용하는 방법을 제안하고, 현재 서비스되는 하스스톤 게임에 적용하여 봄으로써 MCTS 알고리즘의 실용성을 검증한다.

Keywords

References

  1. G. Chaslot, S. Bakkes, I. Szita, & P. Spronck, "Monte-Carlo Tree Search: A New Framework for Game AI", In AIIDE, 2008.
  2. M. Enzenberger, M. Muller, B. Arneson, and R. B. Segal, "Fuego - An Open-Source Framework for Board Games and Go Engine Based on Monte Carlo Tree Search", IEEE Trans. Comp. Intell. AI Games, Vol. 2, No. 4, pp. 259-270, 2010. https://doi.org/10.1109/TCIAIG.2010.2083662
  3. H. J. van den Herik, "The Drosophila Revisited", Int. Comp. Games Assoc. J, Vol. 33, No. 2, pp. 65-66., 2010.
  4. C. S. Lee, M. H. Wang, G. M. J.-B. Chaslot, J. B. Hoock, A. Rimmel, O. Teytaud, S.-R. Tsai, S.-C. Hsu, and T.-P. Hong, "The Computational Intelligence of MoGo Revealed in Taiwan's Computer Go Tournaments", IEEE Trans. Comp. Intell. AI Games, vol. 1, no. 1, pp. 73-89, 2009. https://doi.org/10.1109/TCIAIG.2009.2018703
  5. A. Rimmel, O. Teytaud, C. S. Lee, S. J. Yen, M. H. Wang, and S.-R. Tsai, "Current Frontiers in Computer Go", IEEE Trans. Comp. Intell. AI Games, Vol. 2, No. 4, pp. 229-238, 2010. https://doi.org/10.1109/TCIAIG.2010.2098876
  6. C. S. Lee, M. Muller, and O. Teytaud, "Guest Editorial: Special Issue on Monte Carlo Techniques and Computer Go", IEEE Trans. Comp. Intell. AI Games, Vol. 2, No. 4, pp. 225-228, 2010. https://doi.org/10.1109/TCIAIG.2010.2099154
  7. G. Chaslot, "Monte-carlo tree search.", Maastricht: Universiteit Maastricht, 2010.
  8. M. Buro, J. R. Long, T. Furtak, and N. R. Sturtevant, "Improving State Evaluation, Inference, and Search in Trick-Based Card Games", in Proc. 21st Int. Joint Conf. Artif. Intell., Pasadena, California, pp. 1407-1413, 2009.
  9. P. I. Cowling, C. D. Ward, E. J. Powley, "Ensemble Determinization in Monte Carlo Tree Search for the Imperfect Information Card Game Magic: The Gathering", IEEE Trans. Comp. Intell. AI Games, Vol. 4, No. 4, pp. 241-257, 2012. https://doi.org/10.1109/TCIAIG.2012.2204883
  10. https://en.wikibooks.org/wiki/A-level_Computing/AQA/Paper_1/Theory_of_computation/Finite_state_machines
  11. http://aigamedev.com/open/article/hfsm-gist/
  12. J. Vermorel, & M. Mohri, "Multi-armed bandit algorithms and empirical evaluation", In Machine learning: ECML 2005, Springer Berlin Heidelberg, pp. 437-448. 2005.
  13. A. Garivier, & E. Moulines, " ", arXiv preprint arXiv:0805.3415. 2008.
  14. Kyung-Min. Seo, and Hae-Sang. Song, "Importance Sampling Embedded Experimental Frame Design for Efficient Monte Carlo Simulation", The Journal of the Korea Contents Association, Vol. 13, No. 4, pp.53-63, 2013. https://doi.org/10.5392/JKCA.2013.13.04.053
  15. C. B. Browne, E. Powley, D. Whitehouse, S. M. Lucas, P. I. Cowling, P.Rohlfshagen, ... & S. Colton, "A survey of monte carlo tree search methods", Computational Intelligence and AI in Games, IEEE Transactions on, Vol. 4, No. 1, pp. 1-43, 2012. https://doi.org/10.1109/TCIAIG.2012.2186810
  16. Young-Wook. Choi, Byung-Doo. Lee, "The best move sequence in playing Tic-Tac-Toe game", Korean Society For Computer Game, Vol. 27, No. 3, pp. 11-16. 2014.
  17. https://en.wikipedia.org/wiki/Monte_Carlo_tree_search
  18. http://www.inven.co.kr/board/powerbbs.php?come_idx=3559&name=subject&keyword=%EA%B0%80%EC%B9%98&l=1807