Storing Data to a Cloud Storage (AWS, GCP, and Azure)

In this recipe, you will use pandas to write to cloud storage such as Amazon S3, Google Cloud Storage, and Azure Blob Storage. Several of the pandas writer functions support writing to cloud storage through the storage_options parameter.

Getting ready

In the Reading data from a URL recipe in Chapter 2, Reading Time Series Data from Files, you were instructed to install boto3 and s3fs to read from AWS S3 buckets. In this recipe, you will be leveraging the same libraries in addition to the needed libraries for Google Cloud Storage (gcsfs) and Azure Blob Storage (adlfs).

To install using pip, you can use this:

>>> pip install boto3 s3fs
>>> pip install google-cloud-storage gcsfs
>>> pip install adlfs azure-storage-blob azure-identity

To install using conda, you can use this:

>>> conda install -c conda-forge boto3 s3fs -y
>>> conda install -c conda-forge google-cloud-storage gcsfs -y
&gt...