What is in-place sharing?
In-place data sharing is a method of sharing data where the data remains in its original location. It is not physically copied or moved to a different location when another data product wants to access it. Instead, access is granted to the data where it resides. This approach contrasts with traditional data-sharing methods such as data copy pipelines, which involve copying the data to a new location.
The way the data is accessed depends on the way the data is stored. If the data is stored in a data lake, you can provide a connection string to the data lake that the consuming data product can use to directly read the data and load it into its processing space (a Python DataFrame or SQL query result). Security can be managed using temporary keys or authenticated service principals.
Azure Databricks has a feature called Delta Sharing. It’s an open protocol and is supported by its Unity Catalog service. The shared metadata is stored in Unity Catalog...
