Persisting data on disk

In this section of this chapter, we will look at how to persist data on disk in three different formats. To persist data means that the data is written to non-volatile storage, like a hard drive, for example, and it is not deleted when the process that wrote it ends its life cycle. We will explore the pickle and shelve modules, as well as a short example that will involve accessing a database using SQLAlchemy, perhaps the most widely adopted ORM library in the Python ecosystem.

Serializing data with pickle

The pickle module, from the Python standard library, offers tools to convert Python objects into byte streams, and vice versa. Even though there is a partial overlap in the API that pickle and json expose, the two are quite different. As we have seen previously in this chapter, JSON is a text format that is human readable, language independent, and supports only a restricted subset of Python data types. The pickle module, on the other hand, is not human readable...