Glossary
ETL stands for extract, transform, load: the process of taking data out of source systems, reshaping it into a consistent form, and loading it somewhere it can be analysed.
Modern practice frequently loads raw data first and transforms it afterwards, which keeps the original available when a definition turns out to be wrong. What stays constant is that data has to be moved and reshaped before it can be compared.
Deciding that an order counts on dispatch rather than on payment is a business decision expressed as code, which is why transformations belong in version control and not in a settings panel.
A source system adds a field, changes a format or starts sending nulls, and the pipeline keeps running while producing subtly wrong figures. Validation on the way in is what turns that into an alert rather than a quarter of bad reporting.
The order of the last two steps. ELT loads raw data first and transforms it in the destination, which keeps the original available when a definition changes. For most current warehouses ELT is the default.
Usually because a source changed and nothing checked. Pipelines fail silently by default, so validation and alerting on the way in are what prevent a quarter of quietly incorrect reporting.
ETL stands for extract, transform, load: the process of taking data out of source systems, reshaping it into a consistent form, and loading it somewhere it can be analysed.
Connected
