Figuring out which quality assurance level your company has
Getting started with a company’s testing requires making a series of decisions that could be hard to change later on. To be able to determine the best system for measuring and delivering quality for a company, you can start by asking some clarifying questions.
I also suggest you ask them whether you are about to join a new company or team to set the basis for your job and understand what to expect of the position and whether you will be able to grow professionally and help the company given its circumstances.
What development process is being followed?
Understanding the development process will help you understand how to test it better. Although companies are tending to move towards agile (for the reasons I explain next), there might also be companies that are perfectly content to be working with the waterfall system.
Waterfall
This sort of process is less common nowadays, but it is still in practice, for example, when systems are delivered in conjunction with hardware, or there are no expectations over new versions.
Testing a waterfall system usually implies managing a lengthy list of tests that are added up per feature and establishing planning for executing all those tests before launching the system to the user. We want to test everything that we can think of, and that can cover something that might already be covered with prior tests.
In these systems, it is especially expensive to find a bug in production as it might give a bad name to the product, could end up being dangerous for the end user, and might require building a new batch containing the product fix and having to re-deliver the product for free. You can see some examples of this in the Further reading section at the end of this chapter.
The tests are usually handled by one or more teams of experts, and they could be separated from the developers that built the deliverable (except for unit testing).
Although this is not so strict in a waterfall approach, starting testing early is always recommended. This is a lesson that we have learned from agile, as the earlier bugs are found, the easier they are to debug.
Agile
Although delivering bugs to the customers is something that could damage a company’s reputation and should be avoided at all costs, companies working in agile can afford to have minor bugs, provided they can be fixed quickly and seamlessly. It is important that the user has a platform to report bugs and issues if some of them reach the users.
Not delivering quickly enough could give advantages to competitors, lose clients’ interest, or even make the product irrelevant to the market. Therefore, the most important thing is to be able to balance the quality of the product with the quality of delivery time. This could mean having fewer but smarter tests. Testing something that was tested before will slow down each release. Since we have more frequent releases this could add up time. That said, every time an issue reaches production, it is important to make sure there is a new test in place that will cover this issue next time, while thinking about other tests we could have missed.
API testing gains importance over end-to-end testing in this sort of system (more on this in Chapter 4, The Secret Passages of the Test Pyramid – The Top of the Pyramid). This does not mean that long end-to-end, exploratory, or regression tests should not be done, but we can make use of the API to cover some of them (as we will see in detail in Chapter 4). Visual testing is also gaining importance in these systems.
To achieve more testing, the focus turns to deployment, as we could use servers that are out of reach for the users to do more extensive testing before deploying to them.
The relationship between developers building the feature and developers that are building automation or other members of the test team should be tight in this sort of system to avoid test repetition.
Fake agile
Many companies have moved from a waterfall approach to a fast waterfall that looks a lot like ‘agile,’ but it is not really ‘agile.’ In the process of the movement, we can see a lot of testing being carried over and trying to fit it into a smaller period for sign-off before release. These companies can be identified by having an impatient scrum expert or a project manager asking whether the test case was done and signed off every 5 minutes after the deployment had finished.
The best solution for this is to work alongside dev-ops or developers in charge of the deployments to set up a server system that would allow for more intensive, safer testing and figures out a way of cleaning the list of existing tests. In other words, we want to turn this into an agile system.
What coding language to use?
Unit tests should generally live in the same project of the class they test, using the same programming language.
If there are no tests in place yet, the advice would be to implement them in the same programming language as the developers are using. The reason is that if you turn to combined engineering or shift left in the future, the developers can take ownership of the tests. Sometimes this is difficult because you find that they are using different languages for the frontend and backend.
If there are any tests in place, the best thing is to follow along with what is there unless there is a major reason to rewrite them.
If there are too many tests in place that are not well maintained, then there are two ways of approaching this; refactoring or writing from scratch. Depending on how badly or well-kept the current tests are and how reliable their executions are, it might be easier to do the former or the latter.Here is a graph to help you decide which test code programming language to use:
Figure 1.1: Identify the most suitable test code programming language for you
The way the code is shared and controlled also tells you a lot about the system required, as does the task management, bug tracking, and logging or dashboarding systems.
Finally, if technologies used for development are changing, it would be a good moment to change the testing approach, and it is also crucial to have stable tests to make sure these changes do not affect the users.
What is the relationship between developers and testers?
As discussed in the previous sections, if developers and testers are well aligned, test repetition decreases, but independence might be affected.
Do they sit together in the office?
Being physically near shows that the company cares for communication between development and testing. If the teams sit separately, this could start to form an us and them mentality that could be seen in the product afterward.
For companies that allow remote working, the question would be whether they are in different teams or under the same manager.
Do they perform common code reviews?
Having a common code review system could result in finding bugs earlier. However, there is a general concern that having too many people signing off on a code review might slow it down or that the test team might not understand the code. I suggest that at least the unit tests are reviewed by someone with experience in testing, as code coverage tools could be easily deceived.
This also can have other benefits as all team members can improve their coding and testing skills.
What is the size of the company?
Bigger and well-established companies can afford to have specific teams or people dedicated to tool creation, automation, and quality.
On the other hand, startups struggle most to budget dedicated experts and prefer to find people who can work in multiple areas and adapt to each situation. They might prefer to deliver more bugs in exchange for quicker proof of concepts and faster development. You could argue that developing bugs and having to spend time figuring out what is wrong and where, is not at all faster than developing with superior quality from the get-go, but some of these companies have few, if any, users and are working iteratively to find the right product rather than focusing on building a product right.
The right time for a startup or product to start integrating testing earlier is when the product is about to be launched to users or when the focus changes to building the product right. Unfortunately, by this time, it might be difficult for a tester to start working on automation, as they did not participate in the feature definition and development plan, which could result in testability problems. It will also be harder to write the unit tests then. Therefore, the advice is always to incorporate testing into the development, make it a habit, and get the right set of code reviews and structural automation in place. In this way, the time added will be unnoticeable. We will talk more about unit testing, testability, and the benefits of developing with a test mentality in the next chapter.
Who writes unit tests?
This question might sound redundant, as nowadays test-driven development (TDD) is quite common, and most people understand the importance of unit testing and finding issues as early as possible. Nonetheless, this is not always granted, and some companies prefer someone other than the person that built the code to write all code for testing, including unit testing if needed. If that is the case, the test framework should include unit testing, and these tests might reside in a different place than the feature code.
What are the key test roles?
Some companies might have quality assurance (QA) people, test automation roles, and tool creators, or it might all be done by the developers, with sometimes QA being done by the product owners.
Not all companies assign the name software developer engineer in test (SDET) or QA to the same role. Therefore, it is important that you understand what each role involves and what the expected responsibilities are to be able to help achieve better quality.
It is also important to understand the number of people that are working exclusively on quality versus development, to understand the amount of automation that will be needed.
What does the app or company do?
This might seem obvious, but sometimes it is taken for granted. Depending on what the app or company does, you can think of the right and highest priority type of testing for it. For example, if the app deals with payments, localization and currency, testing is crucial.
How many users do you have or plan to have?
The current versus potential growth is as important for testing as it is for development. Understanding the amount of testing that might be done and planning the best ways of debugging issues found during testing is not always done the same way for smaller projects as for bigger ones. Similarly, we should plan the right testing environments, which might need dedicated servers or other resources. There could be specific problems, such as multi-engagement across users, and performance testing could be of higher importance.
Questions about the development
Having clarity about the way development is done in the company is highly important to create appropriate tests. We looked at development techniques before, but within these techniques, there could be differences in how the features are created. The following questions could also help clarify the quality process.
How frequent are the project iterations?
The more frequent the iterations, the less time to perform testing. If iterations are very frequent and we find the need to do exhaustive testing, we need to be creative about it. For example, having some production servers disconnected for the users with the new code and applying testing there while other servers are being built before distributing them to the users.
How long does it take to verify builds, and what is the goal?
Continuous integration/continuous delivery (CI/CD) is the most automated way of doing this, but the tests could have different degrees of extensiveness depending on the goal.
Is the priority aligned within the company? Per build or per feature?
Is there a clear list of features per build? Is there a straightforward way of tracking which old builds could be affected by new ones? If not, regression tests will need to be more extensive (it is good practice to regression test everything but there is not always enough time for it).
As you can see, successfully defining the right quality process requires a lot of different inputs. There could be more parameters that could help gain a better understanding of the particular needs of a company or project for quality, but by now, you should have a better idea of what works best for each case. In the next section, we will see three examples of common traits that I have seen in the past and analyze how they could be improved. Additionally, you can see a self-assessment in the Appendix.
