Journal of Digital Convergence (디지털융복합연구)

The Society of Digital Policy and Management (한국디지털정책학회)
권호 표지
DOI QR코드

DOI QR Code

Database metadata standardization processing model using web dictionary crawling

웹 사전 크롤링을 이용한 데이터베이스 메타데이터 표준화 처리 모델

  • Jeong, Hana (Department of Computer Engineering, Kongju National University) ;
  • Park, Koo-Rack (Department of Computer Engineering, Kongju National University) ;
  • Chung, Young-suk (Department of Computer Engineering, Kongju National University)
  • 정하나 (공주대학교 컴퓨터공학과) ;
  • 박구락 (공주대학교 컴퓨터공학부) ;
  • 정영석 (공주대학교 컴퓨터공학과)
  • Received : 2021.08.31
  • Accepted : 2021.09.20
  • Published : 2021.09.28
Download PDF
⟨ Previous Next ⟩

Abstract

Data quality management is an important issue these days. Improve data quality by providing consistent metadata. This study presents algorithms that facilitate standard word dictionary management for consistent metadata management. Algorithms are presented to automate synonyms management of database metadata through web dictionary crawling. It also improves the accuracy of the data by resolving homonym distinction issues that may arise during the web dictionary crawling process. The algorithm proposed in this study increases the reliability of metadata data quality compared to the existing passive management. It can also reduce the time spent on registering and managing synonym data. Further research on the new data standardization partial automation model will need to be continued, with a detailed understanding of some of the automatable tasks in future data standardization activities.

데이터 품질 관리는 최근 중요한 이슈로 자리잡았다. 데이터베이스의 메타데이터 표준화는 데이터 품질관리 방안 중 하나이다. 본 연구에서는 일관된 메타데이터 관리를 위하여 표준단어사전 관리를 지원하는 알고리즘을 제시한다. 해당 알고리즘은 웹 사전 크롤링을 통해 데이터베이스 메타데이터의 동의어 관리 자동화를 지원한다. 또한 웹 사전 크롤링 과정에서 생길 수 있는 동음이의어 판별 이슈를 해결하여 데이터의 정확도를 향상시킨다. 본 연구에서 제안하는 알고리즘은 기존의 수동적 관리에 비해 메타데이터 데이터 품질의 신뢰도를 높인다. 또한 이음동의어 데이터 등록 및 관리에 소비되는 시간을 단축시킬 수 있다. 새로운 데이터 표준화 부분 자동화 모델에 대한 추가 연구는 향후 데이터 표준화 프로세스에서 자동화 가능한 작업을 파악하여 진행되어야 한다.

Keywords

References

  1. Janssen, M., Charalabidis, Y., & Zuiderwijk, A. (2012). Benefits, adoption barriers and myths of open data and open government. Information systems management, 29(4), 258-268. DOI : 10.1080/10580530.2012.716740
  2. Pitt, M. A., & Tang, Y. (2013). What should be the data sharing policy of cognitive science?. Topics in Cognitive Science, 5(1), 214-221. DOI : 10.1111/tops.12006
  3. Birney, E., Hudson, T. J., Green, E. D., Gunter, C., Eddy, S., Rogers, J., ... & Yu, J. (2009). Prepublication data sharing. Nature, 461(7261), 168-170. DOI : 10.1038/461168a
  4. Saha, B., & Srivastava, D. (2014, March). Data quality: The other face of big data. In 2014 IEEE 30th international conference on data engineering (pp. 1294-1297). IEEE. DOI : 10.1109/ICDE.2014.6816764
  5. SEnglish, L. P. (1999). Improving data warehouse and business information quality: methods for reducing costs and increasing profits. John Wiley & Sons, Inc.
  6. Haug, A., Zachariassen, F., & Van Liempd, D. (2011). The costs of poor data quality. Journal of Industrial Engineering and Management (JIEM), 4(2), 168-193. DOI : 10.3926/jiem.2011.v4n2.p168-193
  7. Kim, W., & Choi, B. (2003). Towards Quantifying Data Quality Costs. J. Object Technol., 2(4), 69-76. https://doi.org/10.5381/jot.2003.2.4.c6
  8. Eppler, M., & Helfert, M. (2004, November). A classification and analysis of data quality costs. In International Conference on Information Quality (pp. 311-325).
  9. Wang, R. Y., Storey, V. C., & Firth, C. P. (1995). A framework for analysis of data quality research. IEEE transactions on knowledge and data engineering, 7(4), 623-640. DOI : 10.1109/69.404034
  10. Lawrence, R., & Barker, K. (2001, March). Integrating relational database schemas using a standardized dictionary. In Proceedings of the 2001 ACM symposium on Applied computing (pp. 225-230). DOI : 10.1145/372202.372327
  11. Shrivastava, V. (2018). A methodical study of web crawler. Vandana Shrivastava Journal of Engineering Research and Application, 8(11), 01-08. DOI : 10.9790/9622-0811010108
  12. Dhenakaran, S. S., & Sambanthan, K. T. (2011). Web crawler-an overview. International Journal of Computer Science and Communication, 2(1), 265-267.
  13. Pal, A., Tomar, D. S., & Shrivastava, S. C. (2009). Effective focused crawling based on content and link structure analysis. arXiv preprint arXiv:0906.5034.
  14. Jamali, M., Sayyadi, H., Hariri, B. B., & Abolhassani, H. (2006, December). A method for focused crawling using combination of link structure and content similarity. In 2006 IEEE/WIC/ACM International Conference on Web Intelligence (WI 2006 Main Conference Proceedings)(WI'06) (pp. 753-756). IEEE. DOI : 10.1109/WI.2006.19
  15. You, F., Gong, H., Guan, X., Cao, Y., Zhang, C., Lai, S., & Zhao, Y. (2018, August). Design of data mining of WeChat public platform based on Python. In Journal of Physics: Conference Series, 1069(1), p. 012017. IOP Publishing. DOI : 10.1088/1742-6596/1069/1/012017