The most common technique is to reduce the precision of the parameters. Instead of storing them as 32-bit floats, we can store them as 16- or 8-bit floats. There have been experiments for using binary parameters, taking only 1 bit to store.
Quantization is often done at the end of training, when converting the model for use on the device. This conversion impacts the accuracy of the model. Because of this, it is very important to evaluate the model after quantization.
Among all the compression techniques, quantization is often the one with the highest impact on size and the least impact on performance. It is also very easy to implement.
