Generating Statistical Measurements
Python is a general-purpose language with statistical modules. A lot of statistical analysis, such as carrying out descriptive analysis, which includes identifying the distribution of data for numeric variables, generating a correlation matrix, the frequency of levels in categorical variables with identifying mode and so on, can be carried out in Python. The following is an example of correlation:
Figure 8.12: Segment numeric data and correlation matrix output
Identifying the distribution of data and normalizing it is important for parametric models such as linear regression and support vector machines. These algorithms assume the data to be normally distributed. If data is not normally distributed, it can lead to bias in the data. In the following example, we will identify the distribution of data through a normality test and then apply a transformation using the yeo-johnson method to normalize the data:
Figure 8.13: Identifying the distribution of the data...
