Extract-Transform-Load (ETL) Pipeline

From GM-RKB
Jump to navigation Jump to search

An Extract-Transform-Load (ETL) Pipeline is a data processing pipeline composed of data extraction jobs, data transformation jobs, and data loading jobs.



References

2019

  • https://blog.panoply.io/17-great-etl-tools-and-the-case-for-saying-no-to-etl
    • QUOTE: ... ETL has been a bedrock process of data analytics and data warehousing since the beginning, but the increased pace of data usage and the nosediving price of storage mean that it’s often necessary these days to get data in front of analysts as quickly as possible. Because the Transform step in an ETL pipeline can often be a chokepoint in the data pipeline, that means that some more modern data warehousing companies are switching to an ELT-based approach, where the transformation step is pushed to the end of the process, or even delayed until the point of query by analysts. ...