Defining data sources and providing reference data

Every data warehousing project, in theory, can be reduced to finding the right data sources and their transformations to achieve the outputs that you want.

In an ideal world, you can always find the data that you need and produce what you want, but life is usually more complicated, as you might not have all the data sources or information that you would like. The reality is that often you need to adapt your goals and see what you can achieve starting from what you have.

In any case, defining your data sources is crucial, as they are what is provided to you, and by writing proper transformations, you can be the best steward of the information contained therein, unless you are in the rare position to change what the source systems collect or to design your data sources.

Defining data sources in dbt

In dbt, you have two proper ways to designate and take into use external data, that is, data that is not created or transformed...