Reading and Writing Files

Reading and writing files is a fundamental step in your data workflows. Your data processing pipelines have both input and output. Learning and knowing how to input and output your data effectively is an essential component for your successful Polars implementations.

Polars provides a way to read and write files while in lazy mode by utilizing the scan and sink methods. They help push down predicates (filters) and projections (column selections) to the scan level and write the output that is larger than RAM streaming results to disk..

In this chapter, you’ll be learning how to read and write various types of file formats.

We will explore the following recipes in this chapter:

• Reading and writing CSV files
• Reading and writing Parquet files
• Reading and writing Delta Lake tables
• Reading and writing JSON files
• Reading and writing Excel files
• Reading and writing other data file formats
• Reading and writing multiple...