Journal of the Korea Society of Computer and Information (한국컴퓨터정보학회논문지)

Korean Society of Computer Information (한국컴퓨터정보학회)
권호 표지
DOI QR코드

DOI QR Code

Performing Data Integration: Handed-code Approach vs. Tool-based Approach

  • Koo, Heung-Seo (Division of Software Convergence, Cheong-Ju University)
  • Received : 2019.05.22
  • Accepted : 2019.06.13
  • Published : 2019.07.31
Download PDF
⟨ Previous Next ⟩

Abstract

Data integration technology is one of the key elements in building data warehouses or big data, and is used to combine data from multiple sources and provide an integrated view to users. Traditionally, the performance of data integration uses a handed-code approach or a tool-based approach that utilizes data integration tools such as ETL. There is a debate about which methods are efficient. This study is conducted to give practitioners preparing for a data integration project an insight into how to perform data integration. This paper examines the views of experts on the controversy over the adoption of ETL tools that have been on the agenda of the data integration area for over a decade.

Keywords

References

  1. Data Integration, https://en.wikipedia.org/wiki/Data_integration, last edited on 2019.05.01.
  2. ETL: What it is and why it matters, https://www.sas.com/en_us/insights/data-management/what-is-etl.html, accessed on 2019.03.10.
  3. Sung-Ho Shin, Min-Ho Lee, et. al., "A Data Migration Model and Case Study for Building Management System of Science and Technology Contents", Journal of the Korea Society of Computer and Information, Vol.16, No.11, Nov., 2011.
  4. Hee-Seo Park, Hee-Chern Kim, "A Data Migration Method for Developing GIS-based Fisheries Resources Information Systems", Journal of Digital Convergence, Vol.11, No.6, Jun. 2013.
  5. Jong-Keun Choi, Assessment of Open Source ETL Tools for Data Migration, Soongsil Univ. Master's Thesis, 2011.6.
  6. Sang-Hyun Park, A Study on Performance Increase of ETL Data Migration, Soongsil Univ. Master's Thesis, 2015.6.
  7. David Aldridge, A List: Ten Reasons Not To Use An External ETL Tool, Blog in The Oracle Sponge, https://oraclesponge.wordpress.com/2006/12/20/a-list-ten-reasons-not-to-use-an-external-etl-tool/, 2006.12.20.
  8. Madhu Zode, The Evolution of ETL: From Hand-coded ETL to Tool-based ETL, White Paper, Cognizant Technology Solutions, 2007.
  9. Craig Stedman, Automated data integration tools versus manual coding, Decision time, SearchDataManagement.com, 2012.08.12.
  10. Gary Nissen, Is Hand-Coded ETL the Way to Go? Absolutely Yes, or Absolutely No, Depending, Article, http://www.garynissen.com/etl-hand-code-or-tool/, 2013.02.20.
  11. Francisco Blanes Martin-Portugues, ETL tools vs Hand-coded ETL, https://www.linkedin.com/, 2015. 07.31.
  12. Ashley Stirrup, Hand Coding vs. Tools: Our Take on Gartner's Report, Blog in Talend, https://www.talend.com/, 2016.10.12.
  13. Slawomir Chodnicki, To code, to ETL, or to SQL?, blog in Twineworks, Twineworks, https://blog.twineworks.com/are-we-doing-data-pipelines-wrong-4176b5f7964f, 2017.12.05.
  14. Swatee Chand, What are the advantages of using an ETL tool vs. handed coding? And is Talend a good ETL tool?, Quora.com, 2018.02.02.
  15. June Tong, ETL Tools vs. Hand Written SQL, White Paper, Sesame Software, accessed in 2018.8.16.
  16. Timothy King, Data Integration vs. Data Migration: What's the difference?, Solutions Review, https://solutionsreview.com/data-integration-vs-data-migration-what.html, accessed on 2018.08.16.
  17. Margaret Rouse, Data Migration, Blog in SearchCIO, posted 2016.10.14.
  18. Panos Vassiliadis, Alkis Simitsis, "Extraction, Transfor mation, and Loading", Encyclopedia of Database Systems, 2009.
  19. Aman Partap Singh Pall, Jaiteg Singh, "ETL Methodo logies, Limitations and Framework for The Selection and Development of an ETL Tool", International Journal of Research in Engineering and Applied Sciences, Vol.6, No.5, 2016.05.
  20. Scott Henry, et al., "Engineering Trade Study: Extracrt, Transform, Load Tools for Data Migration", Proceedings of the 2005 Systems and Information Engineering Design Symposium, IEEE, pp.1-8, 2005.