Indexing data using Apache Spark

Now that we have installed Apache Spark, we can configure it to work with Elasticsearch and write some data in it.

Getting ready

You need an up-and-running Elasticsearch installation, as we described in the Downloading and installing Elasticsearch recipe in Chapter 1, Getting Started.

You also need a working installation of Apache Spark.

To simplify the configuration, we disable the HTTP Secure Sockets Layer (SSL) self-signed certificate that updates the section of config/elasticsearch.yml to false, as shown in the following code:

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: false
  keystore.path: certs/http.p12

After changing the configuration, the Elasticsearch node/cluster must be restarted.

How to do it...

To configure Apache Spark to communicate with Elasticsearch, we will perform the following steps:

1. We need to download...