Profiling table statistics: Data Profiling Task
Data Profiling is an increasingly used task, and as such, it's necessary to understand the data extracted from sources in detail and also to reduce problems with the data's quality during the data integration process. In an ETL scenario, this data profiling should be done before the ETL process accesses the source system directly, or if that's not possible, then after the Extract stage where there usually isn't any transformation on data applied. If the data profile is not known, the risk of problems during later stages (Transform and Load) will significantly increase.
When source data is profiled using this task, the output could be a column null ratio, column lengths and values distribution, column pattern, and also the candidate columns for being keys. The output of this analysis generates XML reports that can be saved to a file or an SSIS variable.
The results gathered by the Data Profiling Task are very useful for tuning a database.
