Declaring a source
Overview
The source.yml
file in dbt project is used to define the source tables that will be transformed by dbt models. This file contains information such as the database name, schema, and corresponding tables for a given source. By using the {{ source() }}
function, you can select from source tables in your models and establish the lineage of your data.
To use dbt transformations, you need to create the source.yml
file. Here is an example of what the source.yml
file might look like for a project that transforms data from a database:
Sources:
- name: klaviyo
database: platformdbtest\_production\_main
schema: platformdbtest\_production
tables:
- name: campaigns
identifier: klaviyo\_klaviyo\_campaign
- name: events
identifier: klaviyo\_klaviyo\_events
- name: square
database: platformdbtest\_production\_main
schema: platformdbtest\_production
tables:
- name: orderlineitems
identifier: square\_square\_orderlineitems
- name: order
identifier: square\_square\_order
- name: location
identifier: square\_square\_location
In this example of a source.yml
file for dbt transformation, there are two specified sources: klaviyo
and square
. The klaviyo
source is in the platformdbtest\_production\_main
database, platformdbtest\_production
schema and has two tables that can be referenced in dbt models for transformations: klaviyo\_klaviyo\_campaign
as campaigns and klaviyo\_klaviyo\_events
as events.
You can use the {{ source() }}
function to select from source tables in dbt models and establish data lineage. For example, if you create a model that references the klaviyo\_klaviyo\_campaign
table from the klaviyo
source, you can track the lineage of the data in your models. Similarly, if you create a model that references the square\_square\_order
table from the square
source, you can establish the data lineage for that table.