How to create a standard Java HTTP Client in ElasticSearch

[box type="note" align="" class="" width=""]This is an excerpt from a book written by Alberto Paro, titled Elasticsearch 5.x Cookbook. This book is your one-stop guide to mastering the complete ElasticSearch ecosystem with comprehensive recipes on what’s new in Elasticsearch 5.x.[/box]

In this article we see how to create a standard Java HTTP Client in ElasticSearch. All the codes used in this article are available on GitHub. There are scripts to initialize all the required data.

An HTTP client is one of the easiest clients to create. It's very handy because it allows for the calling, not only of the internal methods as the native protocol does, but also of third- party calls implemented in plugins that can be only called via HTTP.

Getting Ready

You need an up-and-running Elasticsearch installation. You will also need a Maven tool, or an IDE that natively supports it for Java programming such as Eclipse or IntelliJ IDEA, must be installed. The code for this recipe is in the chapter_14/http_java_client directory.

How to do it

For creating a HTTP client, we will perform the following steps:

For these examples, we have chosen the Apache HttpComponents that is one of the most widely used libraries for executing HTTP calls. This library is available in the main Maven repository search.maven.org. To enable the compilation in your Maven pom.xml project just add the following code:

<dependency>

<groupId>org.apache.httpcomponents</groupId>

<artifactId>httpclient</artifactId>

<version>4.5.2</version>

</dependency>

If we want to instantiate a client and fetch a document with a get method the code will look like the following:

import org.apache.http.*;

Import org.apache.http.client.methods.CloseableHttpResponse; import org.apache.http.client.methods.HttpGet;

import org.apache.http.impl.client.CloseableHttpClient; import org.apache.http.impl.client.HttpClients;

import org.apache.http.util.EntityUtils; import java.io.*;

public class App {

private static String wsUrl = "http://127.0.0.1:9200"; public static void main(String[] args) {

CloseableHttpClient client = HttpClients.custom()

.setRetryHandler(new MyRequestRetryHandler()).build();

HttpGet method = new HttpGet(wsUrl+"/test-index/test- type/1");

// Execute the method.

try {

CloseableHttpResponse response = client.execute(method);

if (response.getStatusLine().getStatusCode() != HttpStatus.SC_OK) {

System.err.println("Method failed: " +

response.getStatusLine());

}else{

HttpEntity entity = response.getEntity();

String responseBody = EntityUtils.toString(entity); System.out.println(responseBody);

}

} catch (IOException e) { System.err.println("Fatal transport error: " + e.getMessage());

e.printStackTrace();

} finally {

// Release the connection. method.releaseConnection();

}

}

}

The result, if the document will be:

{"_index":"test-index","_type":"test- type","_id":"1","_version":1,"exists":true, "_source" : {...}}

How it works

We perform the previous steps to create and use an HTTP client:

The first step is to initialize the HTTP client object. In the previous code this is done via the following code:

CloseableHttpClient client = HttpClients.custom().setRetryHandler(new MyRequestRetryHandler()).build();

Before using the client, it is a good practice to customize it; in general the client can be modified to provide extra functionalities such as retry support. Retry support is very important for designing robust applications; the IP network protocol is never 100% reliable, so it automatically retries an action if something goes bad (HTTP connection closed, server overhead, and so on).
In the previous code, we defined an HttpRequestRetryHandler, which monitors the execution and repeats it three times before raising an error.
After having set up the client we can define the method call.
In the previous example we want to execute the GET REST call. The used method will be for HttpGet and the URL will be item index/type/id. To initialize the method, the code is: HttpGet method = new HttpGet(wsUrl+"/test-index/test-type/1

To improve the quality of our REST call it's a good practice to add extra controls to the method, such as authentication and custom headers.
The Elasticsearch server by default doesn't require authentication, so we need to provide some security layer at the top of our architecture.
A typical scenario is using your HTTP client with the search guard plugin or the shield plugin, which is part of X-Pack which allows the Elasticsearch REST to be extended with authentication and SSL. After one of these plugins is installed and configured on the server, the following code adds a host entry that allows the credentials to be provided only if context calls are targeting that host.
The authentication is simply basicAuth, but works very well for non-complex deployments:

HttpHost targetHost = new HttpHost("localhost", 9200, "http"); CredentialsProvider credsProvider = new BasicCredentialsProvider();

credsProvider.setCredentials(

new AuthScope(targetHost.getHostName(), targetHost.getPort()), new UsernamePasswordCredentials("username", "password"));

// Create AuthCache instance

AuthCache authCache = new BasicAuthCache();

// Generate BASIC scheme object and add it to local auth cache BasicScheme basicAuth = new BasicScheme(); authCache.put(targetHost, basicAuth);

// Add AuthCache to the execution context HttpClientContext context = HttpClientContext.create(); context.setCredentialsProvider(credsProvider);

The create context must be used in executing the call:

response = client.execute(method, context);

Custom headers allow for passing extra information to the server for executing a call. Some examples could be API keys, or hints about supported formats.
A typical example is using gzip data compression over HTTP to reduce bandwidth usage. To do that, we can add a custom header to the call informing the server that our client accepts encoding: Accept-Encoding, gzip:

request.addHeader("Accept-Encoding", "gzip");

After configuring the call with all the parameters, we can fire up the request:

response = client.execute(method, context);

Every response object must be validated on its return status: if the call is OK, the return status should be 200. In the previous code the check is done in the if statement:

if (response.getStatusLine().getStatusCode() != HttpStatus.SC_OK)

If the call was OK and the status code of the response is 200, we can read the answer:

HttpEntity entity = response.getEntity();

String responseBody = EntityUtils.toString(entity);

The response is wrapped in HttpEntity, which is a stream.

The HTTP client library provides a helper method EntityUtils.toString that reads all the content of HttpEntity as a string. Otherwise we'd need to create some code to read from the string and build the string. Obviously, all the read parts of the call are wrapped in a try-catch block to collect all possible errors due to networking errors.