Connecting Snowflake with Apache Spark
Spark is a general-purpose data processing engine that can connect with different technologies. This recipe walks you through downloading the drivers and performing the configuration required to connect Spark with Snowflake. For simplicity, we will be performing the configuration on a standalone version of Spark running on an on-premise machine, but these instructions are portable to Spark clusters running on cloud platforms as well.
Getting ready
You will need to be connected to your Snowflake instance via the Web UI or the SnowSQL client to execute this recipe.
Note
This recipe assumes that Spark is installed and configured on your machine or that you have access to a cloud-based Spark installation.
How to do it
We will locate the JDBC driver and the Snowflake connector for Spark and use them to configure Spark connectivity with Snowflake:
- Our first step will be to identify the correct version of the Snowflake Connector...