산업경영시스템학회지 (Journal of Korean Society of Industrial and Systems Engineering)

  • 제46권2호
  • /
  • Pages.22-31
  • /
  • 2023
  • /
  • 2005-0461(pISSN)
  • /
  • 2287-7975(eISSN)
한국산업경영시스템학회 (Society of Korea Industrial and System Engineering)
권호 표지
DOI QR코드

DOI QR Code

Human-AI 협력 프로세스 기반의 증거기반 국가혁신 모니터링 연구: 해양수산부 사례

A Study on Human-AI Collaboration Process to Support Evidence-Based National Innovation Monitoring: Case Study on Ministry of Oceans and Fisheries

  • 투고 : 2023.01.25
  • 심사 : 2023.03.21
  • 발행 : 2023.06.30
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

Governments around the world are enacting laws mandating explainable traceability when using AI(Artificial Intelligence) to solve real-world problems. HAI(Human-Centric Artificial Intelligence) is an approach that induces human decision-making through Human-AI collaboration. This research presents a case study that implements the Human-AI collaboration to achieve explainable traceability in governmental data analysis. The Human-AI collaboration explored in this study performs AI inferences for generating labels, followed by AI interpretation to make results more explainable and traceable. The study utilized an example dataset from the Ministry of Oceans and Fisheries to reproduce the Human-AI collaboration process used in actual policy-making, in which the Ministry of Science and ICT utilized R&D PIE(R&D Platform for Investment and Evaluation) to build a government investment portfolio.

키워드

과제정보

This study was supported by research funds of "Korea Institute of Marine Science & Technology Promotion" (P-21-PE-CR11) and "Korea Institute of Science and Technology Information" (K-23-L05-C02-S16, K-23-L03-C04-S01).

참고문헌

  1. Ahn, H. and Lim, D.H., A Study on Fraud Detection in the C2C Used Trade Market Using Doc2vec, Journal of The Korea Society of Computer and Information, 2022, Vol.27, No.3, pp 173-182.
  2. Asmussen., C.B. and Moller, C., Smart literature review: A practical topic modelling approach to exploratory literature review, Journal of Big Data, 2019, pp. 6-93.
  3. Angelov, P.P., Soares, E.A., Jiang, R., Arnold, N.I., and Atkinson, P.M., Explainable artificial intelligence: an analytical review, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2021, Vol.11, No.5, pp 1-13. https://doi.org/10.1002/widm.1424
  4. Bedenkov, A., Moreno, C. Agustin, L., Newman, A., Feng, L., and Kostello, G., Customer Centricity in Medical Affairs Needs Human-Centric Artificial Intelligence, Pharmaceutical Medicine, 2021, Vol.35, pp. 21-29. https://doi.org/10.1007/s40290-020-00378-1
  5. Calisto, F.M., Santiago, C., Nunes, N., and Nascimento, J.C., Introduction of human-centric AI assistant to aid to radiologists for multimodal breast image classification, Int. J. Human-Computer Studies, 2021, Vol.150, 102607.
  6. Cao., D.S., Lianga., Y.Z., Xub., Q.S., Zhanga., L.X., Huc, Q.N., and Lia, H.D., Feature importance sampling-based adaptive random forest as a useful tool to screen underlying lead compounds, J. Chemometrics, 2011, Vol.25, pp. 201-207. https://doi.org/10.1002/cem.1375
  7. Chen, B.C., Kang, K.M., and Kim, J.K., Deep Learning-based Text Summarization Model for Explainable Personalized Movie Recommendation Service, Journal of Information Technology Services, 2022, Vol.21, No.2, pp 109-126. https://doi.org/10.9716/KITS.2022.21.2.109
  8. Cheon, K.M. and Yang, J., Explainable AI Application for Machine Predictive Maintenance, Journal of Korean Society of Industrial and Systems Engineering, 2021, Vol.44, No.4, pp. 227-233. https://doi.org/10.11627/jksie.2021.44.4.227
  9. DARPA., Explainable Artificial Intelligence(XAI), 2022, [Online]. Availabe: https://www.darpa.mil/program/explainable-artificial-intelligence.
  10. EC(European Commission), Feasibility study for an EU monitoring mechanism on key enabling technologies, Belgium: European Commission, 2012.
  11. EC, KETs obsevatory phase ii methodology report, Belgium: European Commission, 2017.
  12. EC, Advanced Technologies for Industry, 2022, [Online]. Available: https://ati.ec.europa.eu/.
  13. EC, Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of The Regions: Building Trust in Human-Centric Artificial Intelligence(COM(2019) 168 final), Belgium: European Commission, 2019.
  14. EC, Semantic Text Analysis tool: SeTA, Belgium: European Commission, 2019.
  15. EC, Policy and Investment Recommendations for Trustworthy AI, Belgium: European Commission, 2019.
  16. EC, Sectoral Considerations on the Policy and Investment Recommendations for Trustworthy Artificial Intelligence, Belgium: European Commission, 2020.
  17. EC, Proposal for a Regulation of the European Parliament and of the Council on Machinery Products (COM(2021) 202 final), Belgium: European Commission, 2021.
  18. EC, Proposal for Directive of the European Parliament and of the Council on adapting non-contractual civil liability rules to artificial intelligence(COM(2022) 496 final), Belgium: European Commission, 2022.
  19. Economic Ministers' Meeting 18-3, Government R&D investment innovation plan to support innovative growth, Joint ministry of Korea Government, 2018.
  20. Fabris, F., Doherty, A., Palmer, D., Pedro de Magalhaes, J., and Freitas, A., A new approach for interpreting Random Forest models and its application to the biology of ageing, Bioinformatics, Vol.34, No.14, pp. 2449-2456.
  21. Garcia-Magarino, I., Muttukrishnan, R., and Lloret, J., Human-Centric AI for Trustworthy IoT Systems with Explainable Multilayer Perceptrons, Data Mining for Internet of Things - IEEE Access, 2019, Vol.7, 125562.
  22. Griffiths, T.L. and Steyvers, M., Finding Scientific Topics, PNAS, 2004, Vol.101, pp. 5228-5235. https://doi.org/10.1073/pnas.0307752101
  23. Ha, T., Coh, B.Y., Lee, M., Yun, B., and Chun, H.W., An Exploratory Approach to Discovering Salary-Related Wording in Job Postings in Korea, Journal of Information Science Theory and Practice, 2022, Vol.10, pp. 86-95. https://doi.org/10.1109/ACCESS.2022.3141910
  24. Hall, P., Gil, N., Kurka, M., and Phan, W., Machine Learning Interpretability with H2O Driverless AI, 2022, [Online], Available: https://docs.h2o.ai/driverless-ai/late st-stable/docs/ booklets/MLIBooklet.pdf.
  25. Henckaerts, R., Antonio, K., and Cote, M.P., When stakes are high: Balancing accuracy and transparency with Model-Agnostic Interpretable Data-driven suRRogates, Expert Systems With Applications, 2022, Vol.202, pp 1-13. https://doi.org/10.1016/j.eswa.2022.117230
  26. How, M.L., Cheah, S.M., Khor, A.C., and Chan, Y.J., Artificial Intelligence-Enhanced Predictive Insights for Advancing Financial Inclusion: A Human-Centric AI-Thinking Approach, Big Data and Cogn. Comput., 2020, Vol.4, No.8, doi:10.3390/bdcc4020008.
  27. How, M.L., Cheah, S.M., Chan, Y.J., Khor, A.C., and Say, E.M.P., Artificial Intelligence-Enhanced Decision Support for Informing Global Sustainable Development: A Human-Centric AI-Thinking Approach, Information, Vol.11, No.39, doi:10.3390/ info11010039.
  28. Hu, Y., Boyd-Graber, J., Satinoff, B., and Smith, A., Interactive topic modeling, Mach Learn, 2014, Vol.95, pp. 423-469. https://doi.org/10.1007/s10994-013-5413-0
  29. Hwang, H. and Lee, Y., Semi-Supervised Learning based on Auto-generated Lexicon using XAI in Sentiment Analysis, Proceedings of Recent Advances in Natural Language Processing, 2021, pp 593-600.
  30. IBM, What is human-centered AI?, 2023, [online] https://research.ibm.com/blog/what-is-human-centered-ai.
  31. Jeong, E.H., The Analysis of Changes in East Coast Tourism using Topic Modeling, JKIIECT, 2020, Vol.13, No.6, pp. 489-495.
  32. Kim, D.Y., Jeong, A.Y., and Lee, T.J., Analysis of Malware Group Classification with eXplainable Artificial Intelligence, Journal of The Korea Institute of Information Security & Cryptology, 2021, Vol.31, No.4, pp 559-571.
  33. Kling, R. and Star, S.L., Human Centered Systems in the Perspective of Organizational and Social Informatics, Computers and Society, 1998, pp. 22-29.
  34. Kumar, S.A., Nasralla, M.M., Garcia-Magarino, I., and Kumar, H., A machine-learning scraping tool for data fusion in the analysis of sentiments about pandemics for supporting business decisions with human-centric AI explanations, PeerJ Comput. Sci., 2021, Vol.7, e713 DOI:10.7717/peerj-cs.713.
  35. Lane, J., Let's make science metrics more scientific, Nature, 2010, Vol.464, pp. 488-489. https://doi.org/10.1038/464488a
  36. Lane, J. and Bertuzzi, S., The Star Metrics Project: Current and Future Uses for S&E Workforce Data, 2010, [Online], Available: https://www.nsf.gov/sbe/sosp/workforce/lane.pdf.
  37. Lepri, B., Oliver, N., and Pentland, A., Ethical machines: The human-centric use of artificial intelligence, iScience, 2021, Vol.24, 102249.
  38. Lee, H.S., Kihm, J.Y., Choo, J.G., Stasko, J., and Park, H.S., iVisClustering: An Interactive Visual Document Clustering vis Topic Modeling, Eurographics Conference on Visualization, 2021, pp. 1155-1164.
  39. Lee, T.Y., Smith, A., Seppi, K., Elmqvist, N., Boyd-Graber, J., and Findlater, L., The human touch: How non-expert users perceive, interpret, and fix topic models, Int. J. Human-Computer Studies, 2017, Vol.105, pp. 28-42. https://doi.org/10.1016/j.ijhcs.2017.03.007
  40. Lim, J.S., Shin, K.M., Yoon, J.S., and Bae, S.H., Study of US/EU National Innovation Policies Based on Nanotechnology Development, and Implications for Korea, Journal of Information Science Theory and Practice, 2015, Vol.3, No.1, pp. 50-65. https://doi.org/10.1633/JISTaP.2015.3.1.4
  41. Lim, J.S. and Kim, S.G., Quantifying Innovation Characteristics of National R&D Investment through Cross-Cutting Data Analysis: Case Study on NTIS Nanotechnology, Journal of The Korean Operations Research and Management Science Society, 2020, Vol.45, No.2, pp. 13-23.
  42. Lim, J.S., Kim, S.G., Bae, S.H., Kim, K.H., and Won, D.K., Research on Regional Spatial Information Analysis Platform about NTIS Raw Data, Journal of Cadastre & Land InformatiX, 2020, Vol.50, No.2, pp 21-35.
  43. Lim, J.S., Park, J.H., and Kim, S.G., Study on Measurement of National R&D Investment Spillover: Linking Multilateral Big Data, Korea CEO Academy, 2021, Vol.24, No.1, pp 311-330.
  44. Lim, J.S. and Bae, S.H., Text Based Explainable AI for Monitoring National Innovations, Journal of Korean Society of Industrial and Systems Engineering, 2022, Vol.45, No.4, pp. 1-7. https://doi.org/10.11627/jksie.2022.45.4.001
  45. Park, J.H. and Lim, J.S., National R&D investment performance analysis based on NTIS raw data: Gyeongsangnam-do Smart Farm Industrialization Case, Regional Industry Review, 2022, Vol.45, No.4, pp. 243-263. https://doi.org/10.33932/rir.45.4.12
  46. Messalas, A., Kanellopoulos, Y., and Makris, C., Model-Agnostic Interpretability with Shapley Values, 10th International Conference on IISA, 2019, pp. 1-7.
  47. Naili, M., Chaibi, A., and Ghezala, H., Arabic topic identification based on empirical studies of topic models, INRIA, 2017, Vol.27, hal-01444574v1f.
  48. Noh, S.H., Analysis of Issues Related to Artificial Intelligence Based on Topic Modeling, Journal of Digital Convergence, 2020, Vol.18, No.5, pp. 75-87.
  49. NTIS(National Science & Technology Information Service), 2022, [Online]. Available: https://www.ntis.go.kr.
  50. OECD. R&D Platform for Investment and Evaluation ("R&D PIE"), 2022, [Online], Available: https://oecd-opsi.org/innovations/rd-platform-for-investment-and-evaluation-rd-pie.
  51. Park, D.J., Oh, H.S., Kim, H.G., and Y, M., Topic Modeling Analysis Comparison for Research Topic in Korean Society of Industrial and Systems Engineering: Concentrated on Research Papers from 1978- 1999, Journal of Korean Society of Industrial and Systems Engineering, 2021, Vol.44, No.4, pp. 113-127. https://doi.org/10.11627/jksie.2021.44.4.113
  52. Park, J.D., A Study on Issues Tracking on Multi-cultural Studies Using Topic Modeling, Journal of the Korea Society for Library and Information Science, 2019, Vol.53, No.3, pp. 273-289.
  53. Schonderwoerd, T.A.J., Jorritsma, W., Neerincx, M.A., and Bosch, K.V.D., Human-centered XAI: Developing design patterns for explanations of clinical decision support systems, Int. J. Human-Computer Studies, 2021, Vol.154, p. 102684.
  54. Smith, A., Kumar, V., Boyd-Graber, J., Seppi, K., and Findlater, L., Closing the Loop: User-Centered Design and Evaluation of a Human-in-the-Loop Topic Modeling System, IUI 2018(Session 3B: Interactive Machine Learning and Analysis), 2018, pp. 293-304.
  55. Stanford University HAI, How will AI change jobs?, 2023, [Online], https://hai.stanford.edu/.
  56. Steyvers, M. and Griffiths, T., Probabilistic topic models, 2007, In Handbook of Latent Semantic Analysis; Lawrence Erlbaum Associates, Inc.: Mahwah, NJ, USA, pp. 424-440.
  57. Yoon, S.U. and Kim, M.C., Topic Modeling on Fine Dust Issues Using LDA Analysis, Journal of Energy Engineering, 2020, Vol.29, pp. 23-29.