과제정보
This work was supported by the faculty research fund of Sejong University in 2023.
참고문헌
- Devlin, Jacob, et al. "Bert: Pre-training of deep bidirectional transformers for language understanding," arXiv preprint arXiv:1810.04805, 2018. DOI: 10.48550/arXiv.1810.04805
- Raffel, Colin, et al. "Exploring the limits of transfer learning with a unified text-to-text transformer," The Journal of Machine Learning Research 21.1, pp.5485-5551, 2020. DOI: 10.48550/arXiv.1910.10683
- Dura, Davide, "Design and analysis of VLSI architectures for Transformers," Diss. Politecnico di Torino, pp.1-2, 2022.
- Vaswani, A., et al. "Attention is all you need," Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, pp.6000-6010, 2017. DOI: 10.48550/arXiv.1706.03762
- Lan, Zhenzhong, et al. "Albert: A lite bert for self-supervised learning of language representations," arXiv preprint arXiv:1909.11942, 2019. DOI: 10.48550/arXiv.1909.11942
- Lu, Siyuan, et al. "Hardware accelerator for multi-head attention and position-wise feed-forward in the transformer," 2020 IEEE 33rd International System-on-Chip Conference, IEEE, pp.2-3, 2020. DOI: 10.48550/arXiv.2009.08605
- Ye, Wenhua, et al. "Accelerating attention mechanism on fpgas based on efficient reconfigurable systolic array," ACM Transactions on Embedded Computing Systems vol.22, no.6, pp.1-22, 2023. DOI: 10.1145/3549937
- Fang, Chao, et al. "An efficient hardware accelerator for sparse transformer neural networks," 2022 IEEE International Symposium on Circuits and Systems (ISCAS). IEEE, pp.2670-2674, 2022. DOI: 10.1109/ISCAS48785.2022.9937659
- Fang, Chao et al. "An Algorithm-Hardware Co-Optimized Framework for Accelerating N:M Sparse Transformers," IEEE Transactions on Very Large Scale Integration (VLSI) Systems, vol.30, pp.1573-1586, 2022. DOI: 10.1109/TVLSI.2022.3197282
- Tuli, Shikhar, Niraj Kumar Jha, "AccelTran: A Sparsity-Aware Accelerator for Dynamic Inference With Transformers," IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol.42, pp.4038-4051, 2023. DOI: 10.1109/TCAD.2023.3273992
- H. T. Kung, B. McDanel, et al. "Maestro: A Memory-on-Logic Architecture for Coordinated Parallel Use of Many Systolic Arrays," 2019 IEEE 30th International Conference on Application-specific Systems, Architectures and Processors (ASAP), New York, NY, USA, pp.42-50, 2019. DOI: 10.1109/ASAP.2019.00-31
- Ye, Wenhua, et al. "Accelerating attention mechanism on fpgas based on efficient reconfigurable systolic array," vol.22, no.6, pp.1-22, 2023. DOI: 10.1145/3549937
- Bansal, Himanshu, et al. "Wallace tree multiplier designs: a performance comparison," Innov Syst Des Eng, vol.5, no.5, pp.67, 2014. DOI: 10.5120/13825-1414
- Tiwari, Shivangi, et al. "Fpga design and implementation of matrix multiplication architecture by ppi-mo techniques," International Journal of Computer Applications, vol.80, no.1, pp.19-22, 2013. DOI: 10.5120/13825-1414
- Elliott, Desmond, et al. "Multi30k: Multilingual english-german image descriptions," arXiv preprint arXiv:1605.00459, pp.73, 2016. DOI:10.18653/v1/W16-3210
- Cettolo, Mauro, et al. "Overview of the iwslt 2017 evaluation campaign," Proceedings of the 14th International Workshop on Spoken Language Translation, pp.4, 2017.
- Wang, Longyue, et al. "Findings of the WMT 2023 Shared Task on Discourse-Level Literary Translation: A Fresh Orb in the Cosmos of LLMs," arXiv preprint arXiv:2311.03127, pp.58, 2023. DOI: 10.48550/arXiv.2311.03127