Load balancers and Splunk

Some organizations that have invested heavily in load balancers like to use them whenever possible to centralize network management. There are three services Splunk typically exposes, mentioned in the following sections:

web

Usually on port 8000, the Splunk web server can be load balanced when configured with search head pooling. The load balancer should be configured to be "sticky", as the web server will still rely on user sessions tied to the web server the user started on.

See the Multiple search heads section for more information.

splunktcp

Usually on port 9997, splunktcp is itself stateless. Splunk auto load balancing is very well tested and very efficient but does not support complicated logic. For instance, you could use a load balancer to prefer connections to indexers in the same data center, only using indexers in another data center as a last resort.

The problem is that when only one address is provided to a Splunk forwarder, the forwarder will open one...