Apache Airflow is an open-source platform for orchestrating, scheduling, and monitoring data pipelines. Originally developed at Airbnb in 2014, it is today one of the most widely used tools in modern data engineering. The key advantage: workflows are defined not as configuration files, but as Python code – meaning they can be versioned, tested, and reused like software.
DAG stands for Directed Acyclic Graph. The term originates from graph theory, but has a central practical significance in data engineering: it describes the dependency structure of tasks or transformation steps that must be executed in a defined order.
Modern data architectures are shifting to the cloud. Classic on-premises data warehouses can reach their limits when faced with growing data volumes, variable load peaks, and the demand for fast deployment cycles. Cloud DWH platforms decouple compute and storage resources, scale elastically, and significantly reduce operational overhead.
dbt (Data Build Tool) is an open-source framework focused on transforming data within the analytics engineering pipeline. Unlike traditional ETL tools that extract, transform and load data, dbt handles only the transformation step directly inside the data warehouse or data lake. The key advantage is that dbt provides SQL-based transformations in a structured, version-controlled and testable format. This allows teams to develop data models like software – with modularity, reusability and automated testing.
Data Vault 2.0 describes a design paradigm for data delivery. A key advantage of Data Vault 2.0 is its ability to respond very flexibly to changes in business processes without fundamentally modifying underlying structures. To achieve this, Data Vault 2.0, in contrast to a traditional data warehouse, is not the defined, aggregated truth of all data but rather the neutral provision of all available data. Data aggregation takes place on top of it, for example in data marts.