Organizing and governing SQL Warehouses

Organizations tend to have a sizeable number of data analysts working on various projects at the same time. Depending on the project, access to different data sources with varying data volumes will have to be accounted for. Depending on the staffing of the project, the number of concurrent users will also need to be accounted for. Depending on the sensitivity of the project, the access control to the work on the project will have to be accounted for. Finally, each project should be held accountable for the computation resources they incur.

The Databricks SQL platform is very flexible and accommodates all these considerations. Let’s start by looking at how SQL Warehouses are distributed to various users or projects.

SQL Warehouse assignment strategy

There are no set rules on how SQL Warehouses should be assigned to users. Databricks certainly does not want to restrict assignment strategies either. There are two levels to the assignment...