Many companies still have discrete test and development teams. This is obviously not an ideal situation, as the test team is usually not completely aware of what the development team is building. This also provides us with additional challenges if the test team is tasked with writing automated functional tests using the web frontend.

The usual problem is that the test team is behind the development team, and how far behind depends upon how frequent development releases are. The thing is, it doesn't really matter how far behind the development team you are. If you are behind the team, you will always be playing catch up. When you are playing catch up, you are constantly updating your scripts to make them work with a new release of software.

What do I mean by living documentation? As the application is built, automated tests are continually being written to ensure that specific criteria are met. These tests come in many different shapes and sizes, ranging from unit tests to integration tests, and leading up to end-to-end functional tests and beyond. All of these tests describe, in some way, how the application works. You have to admit that this sounds just like documentation.

This documentation may not be perfect, but that doesn't stop it from being documentation. Think of an application that has some unit tests, and maybe one badly written end-to-end test. I would equate that to the sort of documentation that you would get with a cheap electrical product, from somewhere such as China. It comes with a manual, which will undoubtedly have a small badly written English bit that doesn...

When it comes to automation, the reliability of tests is the key. If your tests are not reliable, they will not be trusted, which can have far-reaching consequences. I'm sure you have all worked in environments where test reliability has been hard for one of many reasons; let's have a look at a couple of scenarios.

One of the more common reasons that tests are not reliable is having a dedicated test automation team that works in isolation from the team that develops the application. This should really be avoided if possible, as the test automation team is always playing catch up. The development team rolls out new features that the test automation teams...

How can we try to enforce reliability and make sure that these changes are picked up early?

We could ask our developers to run the tests before every push, but sometimes people forget. Maybe they didn't forget, but it's a small change and it doesn't seem worth going through a full test run for something so minor (have you ever heard somebody say, "It's only a CSS change...?"). Making sure that the tests are run, and pass before every push to the centralized source code repository, takes discipline.

What do we do if our team lacks discipline? What if we still keep getting failures that should have been easily caught, even after we have asked people to run the tests before they push code to the central repository? If nothing else works, we could have a discussion with the developers about enforcing this rule.

This is actually surprisingly...

Continuous integration is a way to try and mitigate the issues that we come across by only building and testing code on our development machines. Our continuous integration server will monitor our source code repository and then every time it detects a change, it will trigger a series of actions. The first action will be to build the code, running any tests that it can as it builds the code (usually unit tests), and then creating a deployable artifact. This artifact would then usually be deployed to a server that is a replica of the live environment. Once this code has been deployed to a server, the rest of our tests will be run against that server to ensure that everything is working as expected. If things do not work as expected, the build fails and the development team is notified so that they can fix the problems. It's important to note that...

Since we already have a working Maven implementation, let's enhance it so that it can connect to Selenium-Grid. These enhancements will enable you to connect to any Selenium-Grid, but we are going to specifically look at connecting to a third-party service provided by SauceLabs, since they offer a free tier. Let's have a look at the modifications we need to make to our TestNG code.

We will start off with the modifications to our POM; initially, we are going to add some properties that we can configure on the command line by using this code:

<properties>
     <project.build.sourceEncoding>UTF- 
     8</project.build.sourceEncoding>
     <project.reporting.outputEncoding>UTF-
     8</project.reporting.outputEncoding>
     <!-- Dependency versions -->
     <phantomjsdriver.version>1...

Even if you have made your tests totally reliable, they will fail occasionally. When this happens, it is often very hard to describe the problem with words alone. If one of your tests failed, wouldn't it be easier to explain what went wrong if you had a picture of what was happening in the browser when things went wrong? I know that when any of my Selenium tests fail, the first thing I want to know is what was on the screen at the time of failure. If I knew what was on the screen at the time of failure, I would be able to diagnose the vast majority of issues without having to hunt through a stack trace for a specific line number, and then go and look at the associated code to try and work out what went wrong. Wouldn't it be nice if we got a screenshot showing what was on the screen every time a test failed? Let's take the project...

It's surprising how many people are intimidated by stack traces. A reaction that I regularly see when a stack trace appears on screen is panic!

"Oh my god! Something has gone wrong! There are hundreds of lines of text talking about code I don't recognize and I can't take it all in; what do I do?"

The first thing to do is to relax; stack traces have a lot of information but they are actually really friendly and helpful things. Let's modify our project to produce a stack trace and work through it. We are going to make a small change to the getDriver() method in DriverFactory to force it to always return null by using this code:

    public static WebDriver getDriver() { 
        return null; 
    } 

This is going to make sure that we never return a driver object, something that we would expect to cause...

After reading this chapter, you will hopefully no longer think of automated checks as regression tests; instead you should think of them as living documentation that continues to grow and flourish as the code they are validating changes. When things go wrong you will be able to take screenshots to aid diagnosis, as well as being able to fluently read stack traces. You will know how to connect your tests to a Selenium-Grid to provide additional flexibility. Finally, you will also have a good understanding of why reliability matters and how this feeds into successful continuous integration, continuous delivery, and continuous deployment.

In the next chapter, we are going to have a look at exceptions generated by Selenium. We will work through various exceptions that you may see and what they mean.