You can download the datasets and code for this chapter from the GitHub repository:

• You can find the data here: https://github.com/PacktPublishing/Polars-Cookbook/tree/main/data
• The code is located here: https://github.com/PacktPublishing/Polars-Cookbook/tree/main/Chapter02

Also, it is assumed that you have installed the Polars library in your Python environment:

>>> pip install polars

It is also assumed that you imported it in your code:

import polars as pl

Comma-separated values (CSV) is one of the most commonly used file formats for storing data. The structure or the way in which you read a CSV file may be familiar to you if you have worked with another DataFrame library such as pandas.

In this recipe, we’ll examine how to read and write a CSV file in Polars with some parameters. We’ll also look at how we can do the same in a LazyFrame.

How to do it...

Here are the steps and examples for how to read and write CSV files in Polars:

1. Read the customer_shopping_data.csv dataset into a DataFrame:

   df = pl.read_csv('../data/customer_shopping_data.csv')
   df.head()

   The preceding code will return the following output:

Figure 2.1 – The first five rows of the customer shopping dataset

2. If the CSV file doesn’t have a header, Polars would treat the first row as the header:

   df = pl.read_csv('../data/customer_shopping_data_no_header...

The Parquet file format is an open source columnar file format that’s efficient for data storage and processing. This column-oriented format is suitable for analytics workloads and efficient compression. The Parquet file format is very common in big data analytics.

In this recipe, you will learn how to read and write Parquet files in both a DataFrame and LazyFrame.

Getting ready

Toward the end of the recipe, you’ll need the pyarrow library. If you haven’t yet installed it, run the following command:

>>> pip install pyarrow

How to do it...

We’ll first cover reading a Parquet file:

1. Read a Parquet file:

   parquet_input_file_path = '../data/venture_funding_deals.parquet'
   df = pl.read_parquet(
       parquet_input_file_path,
       columns=['Company', 'Amount', 'Valuation', 'Industry'],
       row_index_name='row_cnt'
   )
   df.head()

   The preceding code...

Delta Lake is an open source storage layer built on top of the Parquet format. Delta Lake has more features than the Parquet format such as versioning and ACID guarantees. It’s basically a Parquet file with some additional benefits.

Many data pipelines nowadays are built in lakehouse architecture, which is a mix of data lakes and warehouses. Delta Lake table is a popular option and is used by many companies. Delta Lake tables can essentially be stored in your data lake but also be queried and used like relational tables. So, Polars being able to work with Delta Lake tables is a big plus.

In this recipe, we’ll look at how to read and write Delta Lake tables with a few useful parameters.

Getting ready

This recipe requires you to install another Python library, deltalake. It’s a dependency required for Polars to work with Delta Lake tables. Run the following command to install it in your Python environment:

&gt...

JavaScript Object Notation (JSON) is an open source file format used to store and transport data. It can easily be parsed into a JavaScript object. JSON is language independent and is used in projects with other programming languages that require a lightweight data exchange format. JSON stores and represents data as key-value pairs. In Python terms, JSON is very much like data that is stored in Python dictionaries.

In this recipe, we’ll cover how to read and write JSON files in Polars. We’ll also cover how to work with a different variation of JSON: Newline Delimited JSON (NDJSON). It is also called JSON Lines (JSONL) or Line-Delimited JSON (LDJSON). As the name suggests, each line is a JSON object.

How to do it...

Next, we’ll dive into how to work with JSON files in Polars:

1. Read a JSON file, showing the first 10 columns:

   df = pl.read_json('../data/world_population.json')
   df.select(df.columns[:10]).head(...

We all know that Excel is one of the most popular data analysis tools out there. It still is the one that most of us are familiar with. Being able to work with Excel in Polars is essential for data analysts. In this recipe, we’ll go through reading and writing Excel files, as well as utilizing some of their useful parameters.

Getting ready

This recipe requires a few Python libraries on top of Polars. You can install it with the following command:

>>> pip install xlsx2csv xlsxwriter

How to do it...

We’ll cover how to read and write Excel files using the following steps:

1. Let’s first read a CSV file into a DataFrame and write it to an Excel file:

   output_file_path = '../data/output/financial_sample_output.xlsx'
   df = pl.read_csv('../data/customer_shopping_data.csv')
   df.write_excel(
       output_file_path,
       worksheet='Output Sheet1',
   &...

There are many other formats aside from the ones introduced earlier. Polars keeps adding features to work with more formats in its frequent updates. We’ll be going over a few other data file formats to read from and write to in Polars.

In this recipe, we’ll cover reading from and/or writing to the Arrow IPC format, Apache Avro, and Apache Iceberg.

These file formats are not as common as the other ones we covered in earlier recipes. However, there are still use cases where companies and people need to work with these formats.

Getting ready

You’ll need to install a few other libraries other than Polars for this recipe. They are pyiceberg, numpy, and pyarrow. Run the following commands in Terminal to install them if you haven’t already:

>>> pip install pyiceberg
>>> pip install numpy
>>> pip install pyarrow

How to do it...

Here are the steps for working with other data...

When working on actual data projects, there are cases where data is split into multiple files in a directory. Dealing with each file one by one can be a pain and may distract you from working on other critical components of your project.

In this recipe, we’ll cover reading multiple files into a single DataFrame or into multiple DataFrames, as well as writing a DataFrame to multiple files.

How to do it...

Here are some ways to work with multiple files:

1. Write a DataFrame to multiple CSV files:
   1. Create a DataFrame:

   data = {'Letter': ['A','B','C'], 'Value': [1,2,3]}
   df = pl.DataFrame(data)

   2. Split it into multiple DataFrames:

   dfs = df.group_by(['Letter'])
   print(dfs)

   The preceding code will return the following output:

   >> <polars.dataframe.group_by.GroupBy object at 0x154373390>

   3. Write them to CSV files:

   for name, df in dfs:
       df.write_csv(f'...

In addition to working with files, it’s very common to work with databases. Polars has integrations with various databases, whether they’re hosted in the cloud or on-premises. Once you understand how to work with one database in Polars, you can apply the same patterns to various other databases.

In this recipe, we’ll specifically look at ways to read from and write to a popular database: Postgres. We’ll look at how to work with cloud databases in Chapter 11, Working with Common Cloud Data Sources.

Getting ready

This recipe requires a few additional dependencies. You’ll need to install the following libraries:

• connectorx
• adbc-driver-postgresql
• pyarrow
• pg8000 (or psycopg2)

You’ll also need to have a Postgres database on your local machine. You can refer to the following websites for more information on how to install a Postgres database locally:

• https://www.postgresql.org/download...