Journal of the Korea Society of Computer and Information (한국컴퓨터정보학회논문지)

Korean Society of Computer Information (한국컴퓨터정보학회)
권호 표지
DOI QR코드

DOI QR Code

A Review of Topological Deep Learning Focused on Simplicial Complex and Cell Complex

  • Ho-Sik Seok (Dept. of Artificial Intelligence and Data Science, Korea Military Academy)
  • Received : 2024.10.24
  • Accepted : 2024.11.18
  • Published : 2024.11.29
Download PDF
⟨ Previous Next ⟩

Abstract

Lots of tasks including physical systems modeling, chemical reaction prediction, and relation extraction require dealing with higher-order relations. Graph neural networks (GNNs) are favorite models for relational data but they have inherent limits due to their focus on pairwise relationships. Topological data analysis (TDA) provides insight into the "shape" of data (or underlying data topology). TDA aims to infer information about data manifold such as connectivity and offers higher-dimensional analog of graphs. Topological deep learning (TDL) combines various deep learning techniques with TDA. TDL enables us to formulate simplicial complex and cell complex through techniques such as low-dimensional embedding and attention. In this paper, we summarize recent achievements especially on simplicial complex and cell complex. We also provide succinct descriptions of related concepts.

화학 반응이나 경제 현상을 모델링할 때는 개체 간의 상호작용을 표현해야 한다. 상호작용의 분석이 필요한 도메인에서 그래프 뉴럴 네트워크를 활발히 적용하고 있으나, 데이터에 존재하는 위상 구조를 표현하는 과정에서 그래프 뉴럴 네트워크의 표현 능력이 충분하지 않다고 알려져 있다. 데이터에 존재하는 위상 구조를 활용하는 토폴로지 데이터 분석(TDA)과 딥러닝을 결합한 토폴로지 딥러닝(TDL)은 복잡한 관계를 내포한 데이터의 분석 및 표현에 강점이 있는데, 본 논문에서는 특히 단체 복합체(simplicial complex)와 셀 복합체(cell complex)에 주목하여 토폴로지 딥러닝의 주요 접근법을 소개하고자 한다. 본 논문에서는 단체 복합체와 셀 복합체를 이해하는 과정에서 요구되는 기본 개념과 주요 접근법을 요약 소개하여 토폴로지 딥러닝을 응용하려는 연구자에게 도움을 제공하고자 한다.

Keywords

Acknowledgement

This study was supported by research fund of Korea Military Academy(Future Strategy and Technology Research Institute). (RN: 24-AI-Center-03).

References

  1. Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and P. S. Yu, "A Comprehensive Survey on Graph Neural Networks," IEEE Trans. Neural Netw. Learn. Syst., Vol. 32, No. 1, pp. 4-24, Jan. 2021. DOI: 10.1109/TNNLS.2020.2978386 
  2. J. Zhou, G. Cui, S. Hu, Z. Zhang, C. Yang, Z. Liu, L. Wang, C. Li, and M. Sun, "Graph Neural Networks: A Review of Methods and Applications," AI Open, Vol. 1, pp. 57-81, Dec 2020. DOI: 10.1016/j.aiopen.2021.01.001 
  3. C. Gao, Y. Zheng, N. Li, Y. Li, Y. Qin, J. Piao, Y. Quan, J. Chang, D. Jin, X. He, and Y. Li, "A Survey of Graph Neural Networks for Recommender Systems: Challenges, Methods, and Directions," ACM Tors, Vol. 1, No. 1, pp. 1-51, Mar 2023. DOI: 10.1145/3568022 
  4. C. Bodnar, "Topological Deep Learning," Ph.D. dissertation, University of Cambridge,
  5. M. Papillon, S. Sanborn, M. Hajij, and N. Miolane, "Architectures of Topological Deep Learning: A Survey of Message-Passing Topological Neural Networks," arXiv https://doi.org/10.48550/arXiv.2304.10031, 2024. 
  6. L.-H. Lim, "Hodge Laplacians on Graphs," SIAM Review, Vol. 62, No. 3, pp. 685-715. Aug 2020. DOI: 10.1137/18M1223101 
  7. M. T. Schaub, A. R. Benson, P. Horn, G. Lippner, and A. Jadbabaie, "Random Walks on Simplicial Complexes and The Normalized Hodge 1-Laplacian," SIAM Review, Vol. 62, No.2, pp. 353-391, May 2020. DOI: 10.1137/18M1201019 
  8. Y. Chen, Y. R. Gel, and H. V. Poor, "BScNets: block simplicial complex neural networks," Proceedings of the 36th AAAI Conference on Artificial Intelligence, pp. 6333-6341, virtual, 2022. DOI: 10.48550/arXiv.2112.06826 
  9. F. Battiston, G. Cencetti, I. Iacopini, V. Latora, M. Lucas, A. Patania, J.-G. Young, and G. Petri, "Networks Beyond Pairwise Interactions: Structure and Dynamics," Physics Reports, Vol. 874, pp. 1-92, Aug 2020. DOI: 10.1016/j.physrep.2020.05.004 
  10. T. M. Roddenberry, N. Glaze, and S. Segarra, "Principled simplicial neural networks for trajectory prediction," Proceedings of the International Conference on Machine Learning. pp. 9020-9029, virtual, 2021. DOI: 10.48550/arXiv.2102.10058 
  11. C. Bodnar, F. Frasca, Y. G. Wang, N. Otter, G. Montufar, P. Lio, and M. Bronstein, "Weisfeiler and Lehman go topological: message passing simplicial networks," Proceedings of International Conference on Machine Learning, pp. 1026-1037, virtual, 2021. DOI: 10.48550/arXiv.2103.03212 
  12. C. Morris, N. M. Kriege, F. Bause, K. Kersting, P. Mutzel, and M. Neumann, "TUDataset: A collection of benchmark datasets for learning with graphs," Proceedings of International Conference on Machine Learning 2020 workshop "Graph Representation Learning and Beyond", virtual, 2020. DOI: 10.48550/arXiv.2007.08663 
  13. R. Eletreby, Y. Zhuang, K. M. Carley, O. Yagan and H. V. Poor, "The Effects of Evolutionary Adaptations on Spreading Processes in Complex Networks," Proc. Natl. Acad. Sci. U.S.A., Vol. 117, No. 11, pp. 5664-5670, Jan 2020. DOI: 10.1073/pnas.1918529117 
  14. M. Salathe, M. Kazandjieva, J. W. Lee, P. Levis, M. W. Feldman, and J. H. Jones, "A High-Resolution Human Contact Network for Infectious Disease Transmission," Proc. Natl. Acad. Sci. U.S.A., Vol. 107, pp. 22020-22025, Nov 2010. DOI: 10.1073/pnas.1009094108 
  15. M. Yang, E. Isufi, M. T. Schaub, and G. Leus, "Simplicial Convolutional Filters," IEEE Trans. Signal Process., Vol. 70, pp. 4633-4648, Sep 2022. DOI: 10.1109/TSP.2022.3207045 
  16. C. W. J. Goh, C. Bodnar, and P. Lio, "Simplicial attention networks," Proceedings of the ICLR 2022 Workshop on Geometrical and Topological Representation Learning, virtual, 2022. DOI: 10.48550/arXiv.2204.09455 
  17. J. Yang and J. Leskovec. "Defining and Evaluating Network Communities based on Ground-Truth," Knowl. Inf. Syst. Vol. 42, pp. 181-213 Jan 2015. DOI: 10.1007/s10115-013-0693-z 
  18. R. Yang, F. Sala, P. Bogdan, "Efficient representation learning for higher-order data with simplicial complexes," Proceedings of the First Learning on Graphs Conference, pp. 13:1-13:21, virtual, 2022. DOI: 10.48550/arXiv.2103.04046 
  19. M. Yang and E. Isufi, "Convolutional Learning on Simplicial Complexes," arXiv preprint https://doi.org/10.48550/arXiv.2301.11163 2023. 
  20. H. Madhu and S. P. Chepuri, "TopoSRL: topology preserving self-supervised simplicial representation learning," Advances in Neural Information Processing Systems 36, pp. 64306-64317, New Orleans, U.S.A. 2023. 
  21. M. Hajij, K. Istvan, and G. Zamzmi, "Cell complex neural networks," Proceedings of NeurIPS 2020 Workshop Topological Data Analysis and Beyond, virtual, 2020. DOI: 10.48550/arXiv.2010.00743 
  22. L. Giusti, C. Battiloro, L. Testa, P. Di Lorenzo, S. Sardellitti and S. Barbarossa, "Cell attention networks," Proceedings of International Joint Conference on Neural Networks, pp. 1-8, Queensland, Australia, 2023. DOI: 10.1109/IJCNN54540.2023.10191530 
  23. P. D. Dobson and A. J. Doig, "Distinguishing Enzyme Structures from Non-Enzymes without Alignments," J. Mol. Biol. Vol. 330, No. 4. pp. 771-783, July 2003. DOI: 10.1016/S0022-2836(03)00628-4 
  24. C. Battiloro, I. Spinelli, L. Telyatnikov, M. Bronstein, S. Scardapane, and P. Di Lorenzo, "From latent graph to latent topology inference: differentiable cell complex module," Proceedings of International Conference on Learning Representation, Vienna Austria, 2024. DOI: 10.48550/arXiv.2305.16174 
  25. B. Peters, V. Niculae, and A. F. T. Martins, "Sparse sequence-to-sequence models," Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1504-1519, Florence, Italy, 2019. DOI: 10.48550/arXiv.1905.05702 
  26. I. Chami, R. Ying, C. Re', and J. Leskovec, "Hyperbolic graph convolutional neural networks," Advances in Neural Information Processing Systems 32, pp. 4868-4879, Vancouver, Canada, 2019. DOI: 10.48550/arXiv.1910.12933 
  27. C. L. Giles, K. D. Bollacker, S. Lawrence "CiteSeer: an automatic citation indexing system," Proceedings of the third ACM conference on Digital libraries, pp. 89-98, Pittsburgh, PA, U.S.A. 19998. DOI: 10.1145/276675.276685 
  28. M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst, "Geometric Deep Learning: Going Beyond Euclidean Data," IEEE Sig. Proc. Mag., Vol. 34, No. 4, pp. 18-42, Jul 2017. DOI: 10.1109/MSP.2017.2693418 
  29. D. Lazer et al., "Life in The Network: The Coming Age of Computational Social Science," Science, Vol. 323, No. 5915, pp. 721-723, Feb 2009. DOI: 10.1126/science.1167742 
  30. R. Lambiotte, M. Rosvall and I. Scholtes, "From Networks to Optimal Higher-order Models of Complex Systems," Nature Physics, Vol. 15, pp. 313-320, Mar 2019. DOI: 10.1038/s41567-019-0459-y 
  31. J. Hoppe and M. T. Schaub, "Representing edge flows on graphs via sparse cell complexes," Proceedings of the Second Learning on Graphs Conference, pp. 1:1-1:22, virtual, 2023. DOI: 10.48550/arXiv.2309.01632 
  32. N. Malod-Dognin and N. Przulj, "Functional Geometry of Protein Interactomes," Bioinformatics, Vol. 35, No. 19, pp. 3727-3734, Mar 2019. DOI: 10.1093/bioinformatics/btz146