dbt

What is it?

dbt (data build tool) is the T in ELT: it takes data already sitting in your warehouse and turns it into clean, analysis-ready models. Each model is a .sql file versioned in git, paired with declarative tests (uniqueness, not-null, referential integrity) and auto-generated documentation. The outcome is a lineage graph (DAG) that traces exactly how each column flows from source to report.

What is it used for?

• Organizing and versioning SQL transformations with the same discipline as application code: pull-request reviews, commit history, rollback.
• Catching data quality issues with tests that run on every build before an analyst ever touches the output.
• Generating documentation and lineage automatically: who consumes which table and where every field comes from.

When to use it / when not to?

Use it when data already lives in your warehouse (BigQuery, Snowflake, DuckDB, Redshift, Trino…) and you want to transform it with engineering discipline: layered, reusable models (staging → intermediate → mart), automated tests, and traceable lineage.

Think twice if what you need is ingestion (moving data from external sources into the warehouse — that is Fivetran, Airbyte, or an Airflow pipeline) or heavy distributed processing outside the warehouse (Spark, Flink). dbt only operates inside the warehouse's SQL engine; it does not replace an orchestrator or an external compute engine.

Get started in 1 minute

pip install dbt-core dbt-duckdb   # swap the adapter for your warehouse
dbt init my_project
cd my_project

Create your first model at models/staging/stg_orders.sql:

-- models/staging/stg_orders.sql
select
    order_id,
    customer_id,
    amount,
    order_date::date as date
from {{ source('raw', 'orders') }}
where amount > 0

Run it:

dbt run          # compiles and materializes models in the warehouse
dbt test         # runs data quality tests
dbt docs serve   # opens the catalog with lineage in your browser