Pickle
The pickle format is Python’s built-in serialization format. Pickle files typically end with a .pkl
extension.
Unlike other formats encountered so far, the pickle format should not be used to transfer data across machines. The main use case is for saving pandas objects that themselves contain Python objects to your own machine, returning to them at a later point in time. If you are unsure if you should be using this format or not, I would advise trying the Apache Parquet format first, which covers a wider array of use cases.
Do not load pickle files from untrusted sources. I would generally only advise using pickle for your own analyses; do not share data or expect to receive data from others in the pickle format.
How to do it
To highlight that the pickle format should really only be used when your pandas objects contain Python objects, let’s imagine we decided to store our Beatles data as a pd.Series
of namedtuple
types. It is a fair question...