Karate in Docker and CI/CD pipelines

As we have seen in the previous chapters, Karate is very powerful for API testing. However, so far, we have only run tests on our local systems. Also, these tests were run by us on an ad hoc basis. These are very helpful while developing applications or running tests when we want some information about the current state of APIs.

In many cases, this is not enough. Instead, these tests should be run automatically within build pipelines that typically pull the current application code and build and deploy it to a test server, test it, and continue deployment until this reaches live instances to be used in production. This process is called continuous integration/continuous deployment (CI/CD), with the goal of a completely automated build and test flow.

In any case, this must be all triggered by automation, so we don’t forget to run tests at the appropriate time. This does not mean, though, that we should not be informed about the test results if there are any problems. We could even go so far as to stop further deployment to live servers in case of issues on our test instances. Also, it is helpful if our Karate tests are run regularly against live instances of an application so we can use it for monitoring purposes.

Shift left testing

In recent times, the concept of shifting left has gained traction. This means that tests should be run at the earliest time possible. In the case of API tests, that means that they should be run as soon as a new version of an API is available on a test server. This way, it is much easier to track down what has caused issues when tests fail. Also, the correct team dealing with API development can be notified automatically and directly without delay. If all tests were run only after every part of a system was built and deployed, it would take much longer to determine which team is responsible for fixing bugs and what component caused the error.

This is usually done is by specialized build servers such as Jenkins or online platforms such as GitHub that offer this functionality.

In this chapter, we will explore different ways to run our test to suit these platforms and fulfill our requirements. It is not possible in this limited scope to give you a full step-by-step guide on how to reach a complete CI/CD setup. Rather, it is intended as a starting point to provide ideas and further learning on how to achieve this goal.

The main topics covered in this chapter are the following:

• Triggering Karate tests from shell scripts
• Running Karate tests in a Docker container
• Customizing our tests
• Integrating Karate tests into GitHub workflows