Communications for Statistical Applications and Methods

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

Supervised text data augmentation method for deep neural networks

  • Jaehwan Seol (Department of Statistics, Pusan National University) ;
  • Jieun Jung (Department of Statistics, Pusan National University) ;
  • Yeonseok Choi (Department of Statistics, Pusan National University) ;
  • Yong-Seok Choi (Department of Statistics, Pusan National University)
  • Received : 2022.11.04
  • Accepted : 2023.01.26
  • Published : 2023.05.31
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, there have been many improvements in general language models using architectures such as GPT-3 proposed by Brown et al. (2020). Nevertheless, training complex models can hardly be done if the number of data is very small. Data augmentation that addressed this problem was more than normal success in image data. Image augmentation technology significantly improves model performance without any additional data or architectural changes (Perez and Wang, 2017). However, applying this technique to textual data has many challenges because the noise to be added is veiled. Thus, we have developed a novel method for performing data augmentation on text data. We divide the data into signals with positive or negative meaning and noise without them, and then perform data augmentation using k-doc augmentation to randomly combine signals and noises from all data to generate new data.

Keywords

References

  1. Brown TB, Mann B, Ryder N et al. (2020). Language models are few-shot learners, Available from: arXiv preprint arXiv:2005.14165
  2. Katharopoulos A and Fleuret F (2018). Not all samples are created equal: Deep learning with importance sampling, Available from: arXiv preprint arXiv:1803.00942
  3. Kay SM (1993). Fundamentals of Statistical Signal Processing, Prentice Hall PTR, NewJersy.
  4. Kim Y (2014). Convolutional neural networks for sentence classification, Available from: arXiv preprint arXiv:1408.5882
  5. Lehrer R (2017). Modeling signal-noise processes supports student construction of a hierarchical image of sample, Statistics Education Research Journal, 16, 64-85. https://doi.org/10.52041/serj.v16i2.185
  6. Maas A, Daly RE, Pham PT, Huang D, Ng AY, and Potts C (2011). Learning word vectors for sentiment analysis, In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, Oregon, USA, 142-150.
  7. Perez L and Wang J (2017). The effectiveness of data augmentation in image classification using deep learning, Available from: arXiv preprint arXiv:1712.04621 1712.
  8. Snell J, Swersky K, and Zemel R (2017). Prototypical networks for few-shot learning, Available from: arXiv preprint arXiv:1703.05175
  9. Wei J and Zou K (2019). Eda: Easy data augmentation techniques for boosting performance on text classification tasks, Available from: arXiv preprint arXiv:1901.11196