Introduction to Automated Testing

Testing is a fundamental task in software development. Even if you consider yourself a bad tester, or even a bad developer, you do some testing when you code your app. At the very least, you open the app to see whether it works as expected.

Maybe you are a little bit more methodical and you have a test plan, at least in your mind. You know that when you code a form, you have to validate some common scenarios:

• Try to save a form with empty fields.
• Try to save with good data.
• Try to enter bad data. You might enter text in numeric fields, invalid dates, and so on.

More experienced developers will cover all the possible scenarios. They will write code based on those scenarios and then test accordingly.

Then we get to the word that's driving this book: we automate stuff. We want to automate our tests. We don't want to forget any scenarios or have to test the same thing over and over.

As you will notice, I haven't mentioned Quality Assurance (QA) analysts yet, because I want to highlight that testing is not something relegated to the QA team. Those who are involved in the testing process include the following:

• Backend developers
• Frontend developers
• QA analysts
• Managers (product or project managers)

We need to know that there are different types of tests. Some types of tests will be performed by developers and QA analysts. Other tests will be specific to either developers or QA analysts.

Mike Cohn, in his book Succeeding with Agile (Addison-Wesley Professional), introduced his very popular Testing Pyramid:

Mike Cohn's Testing Pyramid

Although Mike's book is more than 10 years old, this pyramid is still valid.

This pyramid is based on three characteristics:

• Number of tests
• Isolation
• Speed

I have only one thing against this pyramid: the word UI. Modern apps rely more and more on client code, "UI" code. Frameworks such as React, Angular, and Vue.js allow developers to write reusable components. Many apps now have most of their business rules running on the client.

Frontend developers should not be limited to the top of this pyramid. They should be able to write unit tests and service tests for their UI code. This might look like a small change, but I think it's important. With this change in the paradigm, we get a pyramid that looks like this:

New Pyramid

Now that we have a better understanding, let's talk about the different levels of this pyramid.

Unit tests

Unit tests are the base of the pyramid. The more business logic you cover in unit tests, the less ground you will need to cover in service or UI tests.

As we can see in the pyramid, unit tests need to be fast and isolated. That means that a good unit test shouldn't depend on the environment or any other function. Sometimes this is easier said than done. For instance, if you want to test that the total amount of an invoice is equal to the sum of its items, you should be able to test that specific functionality in the code, without launching a web server or getting data from a database.

What roles use Unit tests?

Backend developers: For sure, Unit tests are for them. They follow the Test-Driven Development (TDD) process if possible. TDD is a technique in software development where tests are written even before any source code has been written. Once the tests have been written, the developer will program the source code to make them pass.

Frontend developers: Writing unit tests was almost impossible in the past. If you didn't have the right tools, you couldn't do your job correctly. But now, many modern libraries support unit testing. If you use React and Redux, you will find that Redux has a way to write unit tests for your components (https://www.hardkoded.com/ui-testing-with-puppeteer/redux-unit-tests).

That's not all. In the same way that backend developers need to think about how to make their code testable, if frontend developers, using modern frameworks, start creating small and testable components, they should be able to use Puppeteer to write UI unit tests. And here is where the "UI" at the top of the testing pyramid stops making any sense. Now we can write UI unit tests.

We can run a small test, rendering a component and testing, for instance, that it "renders a textbox and when I enter a value, the label below changes," or "if I pass a list of 10 items, 10 elements are rendered."

We moved UI testing to the bottom of the testing pyramid.

QA Analysts are not involved yet. Unit tests are about testing the internal code.

How about Managers? If you are a developer, I believe you are going to show this paragraph to your boss. Managers won't write unit tests, but they need to know the importance of writing unit tests and investing time in them.

These are the four benefits you (or your boss) need to know about.

Unit tests show how the code works

Unit tests explain how the code works. When I review code, I start by reviewing unit tests. If I find unit tests saying, for instance, "Create order should send email". I could read that test first, and then, check how that rule was implemented.

Business Analysts or project managers could read these tests and see whether there are any scenarios that haven't been covered or some missing validation.

Unit tests make refactoring possible

I took a risk using the word possible. But I believe that's true. You can't refactor your code if you don't have unit tests backing your changes. Remember, refactoring is changing the implementation of your code without changing the result given specific inputs. Unit tests guarantee that premise.

Unit tests prevent regressions

Regression is an involuntary change in the expected behavior of an app. If we have a good set of tests, they will prevent us from breaking any behavior of the app while we implement new features or while fixing bugs.

How can I make sure that some other developers won't come and break the precious function I just wrote? By writing unit tests. A unit test is a version of you in the future enforcing how a piece of code should work. "Create an order should send an email" – no one will be able to break that rule.

When I review code, changes in unit tests are a red flag to me. I'm not saying that unit tests shouldn't change. But if a test changes, there must be an explanation. Now, the "Create and order should send an email" shows that the sent email count is 2. Is that right? Are we sending another email? Or do we have a regression? Pay attention to changes in unit tests.

Time to go up in the test pyramid.

Service tests

Service tests are also known as Integration Tests. These tests will check how your code interacts with other components. When we talk about components, we are talking about the following:

• Databases
• Other components in the app
• External services

Frontend developers would also need to integrate their code with the following:

• Other UI components
• CSS files
• REST APIs

As we mentioned before, when we go up in the testing pyramid, tests become slower and less stable. And it's supposed to be like that. You will be connecting to a real database or interacting with a real REST API that would use real network calls. That would also mean that your tests would expect the environment to respond in a certain way. For instance, you would expect the database to have some set of data ready to be used, or a REST API to be available.

That's why the more tests you have in the unit test layer, the fewer integration tests you will need to code.

Let's take, for instance, the class that sends an email, could you code an integration test for that? Sure. You set up a local email server that would write emails in a temp folder, so after creating an order, you could check that folder and see whether the email server processed the email your app should have sent. But, as you can see, these kinds of orchestrations are harder to code than small unit tests.

Why do we need integration tests? Why don't we code unit tests only?

Well, you need to tests your integrations. Your code won't run in isolation. If you are testing the backend, you need to see how the database reacts to the data you are inserting, or whether a SQL query returns the data you expect.

If you are a frontend developer, this is where you would invest most of your time, checking how your component interacts on a page or how the HTML being generated affects other elements in the DOM. You would need to test how your component is being rendered with a real REST endpoint, instead of using a dummy JSON file.

What roles use Integration tests?

Backend developers: I've heard people say that these are the only tests that matter. Although I disagree with that strong opinion, I do believe these tests are essential. Say I created a unit test where, for instance, when I call CreateOrder, I get a new Order object. But now, I need to test that when I make a POST request to /orders, an order is created in the database.

Frontend developers will create tests to check how all the different components interact with each other on a page. Again, it's UI testing down in the testing pyramid.

QA Analysts will create tests similar to the tests backend and frontend developers create but with a different perspective.

Developers and QA Analysts create the same kinds of tests but with a different perspective.

Developers will create tests to back their job, so they can check whether they broke anything. And, as we mentioned before, they need tests to be able to refactor their code in the future.

QA Analysts will create tests to guarantee the application quality to the stakeholders.

There is one interesting type of test that QA Analysts can implement in this layer: the Visual Regression Test. These tests are used when we want to check whether there was any visual change regarding the style of the app. We don't want to check whether there is a button, or whether that button works. We want to check whether the button looks like how it was before. How can we achieve that? By comparing images. This technique is based on four steps:

1. We take a screenshot as a baseline:
   

   Baseline image

2. We make a change in the code.
3. We take another screenshot:
   

   Image after making a change

4. We compare both images:

Differences

This type of test can be quite unstable. I bet you have seen that pages sometimes "move" when they are loading, so you have to be very sure when the page is ready for a screenshot. But it is doable. Another downside is that for every error you get, you have to analyze whether the change was a regression (a change made by mistake) or we are in the presence of a new baseline.

The role of managers is still important. They need to provide the tools and the time for developers to implement the required integration tests. They will also help QA Analysts to determine what the integrations to test are.

And so we come to the top of the pyramid, the end-to-end tests.

End-to-end tests

You might also find these tests referred to as E2E tests. The goal of E2E tests is to guarantee that an application works as expected through the entire workflow. Most applications will have more than one workflow. That would mean that it will require a number of E2E tests to cover all the possible workflows or scenarios.

Let's take a cart app as an example. These could be our tests:

• Unit tests:

  a) Passing a cart object, the AddToCart component renders an Add to cart link if the product is not in the array.

  b) Passing a cart object, the AddToCart component renders a "View cart" link if the product is in the array.

• Integration tests:

  a) Go to a product page and click "Add to cart." The link changes to "View cart."

  b) Go to the checkout page. After clicking on the Checkout button, it gets disabled.

• One E2E test testing the cart flow:

  a) Go to a product page, click Add to cart, then click on View cart.

  b) You should have got to the checkout page. Click Checkout.

  c) You should have been redirected to the receipt page.

  d) The receipt should show the product added to the cart.

  e) The price should be the product price.

We are at the top of the pyramid. That means that these will be the slowest and least stable tests.

Why least stable? Check the workflow. Many bad things can happen there. The add to cart endpoint might take a little bit more than expected. The scroll to the Checkout button could have failed for just a few pixels. Your database might be in an unexpected state. Maybe your user already purchased that product, so the Add to cart button is not enabled.

How about roles?

This is the QA Analyst's land. This is where they need to take advantage of all the features Puppeteer provides to make reliable tests. But Developers play an important role, helping the QA team to do their job efficiently. As we are going to see in the next chapters, a developer can leave hints so that the QA team can find the components they need.

I hope the picture of the pyramid makes more sense now. We need lots of small and isolated unit tests, many integration tests testing our pages, and finally, a good set of E2E tests, checking the workflow's health.

This is the famous testing pyramid, but how do we write a test? Where do we write them? How do we run a test?

First, we need to know what we need from a test runner.