What do you get with Print?

Instant access to your digital copy whilst your Print order is Shipped

Paperback book shipped to your preferred address

Redeem a companion digital copy on all Print orders

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

Learning Software Testing with Test Studio

Chapter 1. Introduction

Some of the reasons why you would want to achieve software quality are to make sure that the system does what it's supposed to do; uncover errors and/or to provide assurance for your software user. This chapter will explore some general terminologies and processes in software testing to shed light over some concepts used in this book, and briefly introduce the tool automation features that are covered in the next chapters.

If you are already familiar with the following testing concepts, you can jump to Test Studio uncovered in the later chapters of this book.

Testing concepts

The following is a conceptual overview of some fundamental testing terminologies and principles. These are used in day-to-day testing activities and will be directly referred to in the chapters when explaining the business case for our examples.

Test case

A test case is a scenario that will be executed by the tester or by an automation tool, such as the Test Studio for any of the software testing purposes, such as uncovering potential errors in the system. It contains:

Test case identifier: This identifier uniquely distinguishes a test case.
Priority: The priority holds a value to indicate the importance of a test case so that the most important ones are executed first and so on.
Preconditions: The preconditions describe the initial application state in which the test case is to be executed. It includes actions that need to be completed before starting the execution of the test case, such as performing certain configurations on the application, or other details about the application's state that are found relevant.
Procedure: The procedure of a test case is the set of steps that the tester or automated testing tool needs to follow.
Expected behavior: It is important to set an expected behavior resulting from the procedure. How else would you verify the functionality you are testing? The expected behavior of a test case is specified before running a test, and it describes a logical and friendly response to your input from the system. When you compare the actual response of the system to the preset expected behavior, you determine whether the test case was a success or a failure.

Executing a test case

When executing a test case, you would add at least one field to your test case description. It is called the actual behavior and it logs the response of the system to the procedure. If the actual behavior deviates from the expected behavior, an incident report is created. This incident report is further analyzed and in case a flaw is identified in the system, a fix is provided to solve the issue. The information that an incident report would include are the details of the test case in addition to the actual behavior that describes the anomalous events. The following example demonstrates the basic fields found in a sample incident report. It describes a transaction carried out at a bank's ATM:

Incident report identifier: ATM-398
Preconditions: User account balance is $1000
Procedure: It includes the following steps:
1. User inserts a card.
2. User enters the pin.
3. Attempts to withdraw a sum of $500.
Expected behavior: Operation is allowed
Actual behavior: Operation is rejected, insufficient funds in account!
Procedure results: Fail

The exit criteria

The following definition appears in the ISTQB (International Software Testing Qualification Board) glossary:

"The set of generic and specific conditions, agreed upon with the stakeholders, for permitting a process to be officially completed. The purpose of exit criteria is to prevent a task from being considered completed when there are still outstanding parts of the task, which have not been finished. Exit criteria are used to report against and to plan when to stop testing. [After Gilb and Graham]"

The pesticide paradox

Software testing is governed by a set of principles out of which we list the pesticide paradox. The following definition appears in the ISTQB glossary:

If the same tests are repeated over and over again, eventually the same set of test cases will no longer find any new defects. To overcome this, "pesticide paradox", the test cases need to be regularly reviewed and revised, and new and different tests need to be written to exercise different parts of the software or system to potentially find more defects.

Element recognition

Element recognition is a pillar of automated test execution as the tool used can't perform an action on an object unless it recognizes it and knows how to find it. Element identification is important in making the automated scripts less fragile during execution. This topic will be reflected in this book.

Testing techniques

Ranging from easy and straightforward to complex and machine-computed, many testing techniques guide the design and generation of your test cases. In the this section, we will describe the most basic of these techniques based on the ISTQB standards:

Equivalence classes: By definition, an equivalence class is a single class of inputs generating an equivalent output. Vice versa, it could be a single class of outputs generated from equivalent inputs. For example, imagine you need to test a simple numeric field which accepts values from 0 to 100. During your testing, you cannot possibly exhaust all the values, hence we would identify one valid equivalence partition and three invalid partitions as follows:
For valid partitions:
- Values between 0 and 100 inclusive
For invalid partitions:
- Values less than zero
- Values greater than 100
- Nonnumeric inputs
As a result, you now choose tests from the four equivalence classes instead of testing all the options. The value of equivalence classes analysis lies in the reduction of testing time and effort.
Boundary values: When choosing boundary value analysis, you study the limits of your system input. Typically, they are the logical minimum and maximum values in addition to technical or computational limits, such as register sizes, buffer sizes, or memory availability. After determining your logical and technical limits, you would test the system by inputting the actual boundary, the boundary decremented by the smallest possible unit, and the boundary increment by the smallest possible unit.
Assuming our system is an application form where you need to enter your first name in one of the fields, you can proceed with a boundary value analysis on the length of the first name string. Considering that the smallest input is one character, and the largest input is one hundred, our boundary values analysis will lead to a test for strings having the following number of characters: zero (empty input), one, two, ninety-nine, one hundred, and one hundred and one.
Decision tables: In certain systems, many rules may be interacting with each other to produce the output, such as a security matrix. For instance, let's assume your system is a document management system. The possible factors determining whether a user will have view rights or not are as follows:
- Belonging to user groups with a permission set for each group
- Having an individual permission for each user
- Having access to the documents' file path
These factors are called the conditions of the decision table, where the actions might be reading, editing, or deleting a document. A decision table would allow you to test and verify every combination of the listed conditions. Certain rules might simplify your table, but they are outside the scope of this book. The resulting decision table for the previous example of document management system is illustrated as follows:
Decision table for user rights
State transition diagram: In some systems, not only do the actions performed determine the output and the routing of the application, but also the state in which the system was in before these actions. For such systems, a state transition diagram is used to generate test cases.
1. Firstly, the state transition diagram is drawn with every state as a circle and every possible action as an arrow. Conditions are written between square brackets and the output is preceded by a forward slash.
2. Secondly, each action represented in the diagram is attempted from an initial state.
3. Thirdly, test cases are generated by looping around the state transition diagram and by choosing different possible paths while varying the conditions.
The expected behavior in state transition test cases are both the output of the system and the transition to the next expected state. In the following sample diagram, you will find the state transition diagram of a login module:
State transition diagram for user authentication to the system

Test Studio uncovered through the chapters

This section gives the list of features provided in Test Studio and the chapters in which they are reflected:

Functional test automation: The Test Studio solution to functional test automation is going to be discovered through the following topics: building automated tests, using translators and inserting verifications, adding coded steps, executing tests and logging, adding custom logging, inserting manual steps, assigning and reading variables in tests, debugging errors, and integrating automated test creations with Visual Studio. These topics will be found in Chapter 2, Automating Functional Tests and Chapter 10, Tips and Tricks.
Data-driven architecture: Test Studio offers built-in integration with data sources, allowing you to apply the data-driven architecture during test automation. This feature includes binding tests to SQL, MS Excel, XML, and local data sources, creating data-driven verification, and integrating data-driven architecture with normal automated execution contexts. These topics will be found in Chapter 3, Data-driven Tests and Chapter 10, Tips and Tricks.
Element recognition: Element recognition is a powerful feature in Test Studio from which it derives additional test reliability. Element recognition topics will be covered through Test Studio Find expressions for UI elements, element repository consolidation and maintenance, and specialized Find chained expressions. These topics will be found in Chapter 4, Maintaining Test Elements and Chapter 10, Tips and Tricks.
Manual testing: In addition to automated testing, Test Studio guides the manual testing process. Manual testing includes creating manual test steps, integrating with MS Excel, converting manual tests to hybrid, and executing these two types of tests. These topics will be covered in Chapter 5, Manual Testing.
Organizing the test repository and source control: Tests within the Test Studio project can be organized and reorganized using the features embedded in the tool. Its integration with external source control systems also adds to this management process. The underlying topics are managing tests under folders, setting test properties, and binding your test project to source control from both Test Studio and Visual Studio. The best practices on test repository organization will be encountered throughout the examples of the first four chapters, where the source control topic will be discussed in Chapter 5, Manual Testing since we will have covered all the types of tests offered by the tool by then.
Test suites execution and reporting: Grouping tests under test suites is achievable through the Test Studio test lists. This feature comprises creating static and dynamic test lists, executing them, logging their execution result, viewing standard reports, and extending with custom reports. These topics will be covered in Chapter 6, Test Lists and Reports.
Extended libraries: Extending testing framework automation functionalities for Test Studio is an option available through the creation of Test Studio plugin libraries. This topic will be covered in Chapter 10, Tips and Tricks.
Performance testing: In Test Studio, nonfunctional testing is firstly addressed with performance testing. This feature covers developing performance tests, executing them, gathering performance counters, and analyzing and baselining execution results. These topics will be covered in Chapter 7, Performance Testing.
Load testing: Nonfunctional testing in Test Studio is augmented with another type of test, which is load testing. This topic covers configuring Test Studio load testing services, developing load tests, recording HTTP traffic, creating user profiles and workloads, monitoring machines, gathering performance metrics, executing load tests, and creating custom charts. These topics will be addressed in Chapter 8, Load Testing.
Mobile testing: Test Studio is extended with a version specialized in iOS web, native and hybrid apps testing. It includes preparing applications for testing within Test Studio, creating automated tests, inserting verifications on UI elements, registering applications on the web portal, syncing test projects, sending and viewing built-in feedback messages, sending and viewing crash reports, and managing and monitoring registered applications through web portals. These topics will be addressed in Chapter 9, Mobile Testing.

Setting up your environment

You will get a list of files with this book to help you try the examples properly. The following is an explanation on how to set up the environment to practice the automation examples against the applications under test.

The File Comparer application

Chapters 2 through 6, as well as Chapter 10, Tips and Tricks, revolve around the File Comparer WPF application. To configure this application environment, you need to:

Run the FC_DB-Database Scripts.sql files in the SQL Management Studio.
Open the settings.xml file from the solution bin and edit the ConnectionString parameter.

Reports

Chapter 6, Test Lists and Reports, gives examples on custom reports that can be found in the File Comparer - Reports.xlsx workbook. The data source files for these reports can be found in the ODCs folder. In order to properly display the charts in the workbook:

Edit the ConnectionString parameter inside the ODC extension files.
Bind the pivot tables inside the excel workbook to these files as follows:
- The Execution Metrics for Last Run sheet to the FC_DB-L-EMLR.odc file
- The Execution Metrics over Time sheet to the FC_DB-MOT.odc file
- The Feature Coverage sheet to the FC_DB-FC.odc file
- The Test Execution Duration sheet to the FC_DB-TED.odc file

Alternatively, you can create these charts with the queries provided in Chapter 6, Test Lists and Reports.

Additional files

The following are the additional files used in this book:

The Test Studio Automated Solutions folder contains the Test Studio automated solution for the examples in the book. For each chapter, there will be an incremental solution holding the examples for this chapter and the ones before.
The TestStudio.Extension folder is a Visual Studio solution and it corresponds to the Test Studio extension library demonstrated in Chapter 10, Tips and Tricks.

As for the remaining files, such as the data sources, external tests, and the fiddler SAZ file, they will be referenced by their names within the chapter examples.

Key benefits

Learn to use Test Studio to design and automate tests valued with their functionality and maintainability

Run manual and automated test suites and view reports on them

Filled with practical examples, snapshots and Test Studio hints to automate and substitute throwaway tests with long term frameworks

Description

Test Studio is Telerik's QA solution for automating your manual testing. With Test Studio's standalone edition or Visual Studio plugin, you can rule out the possibility of unreliable test execution and UI recognition, non indicative test results and reports, dispersed test repository, low code coverage, and unaffordable learning curves. With no code, this tool provides an intuitive IDE to effortlessly create maintainable tests. If you are looking for a solution to automate testing for your web, desktop, or mobile application, you can now benefit from Test Studio's rich automation features. "Learning Software Testing with Test Studio" will illustrate how to reliably automate test cases when it is time to relinquish manual testing habits. This book is all about less theory and more hands-on examples to present a complete manual and automated solution for your ASP .NET, WPF, Silverlight or iOS apps. This book gets you started directly with automation in Test Studio by exploiting its recording powers through series of concrete test cases built around the equipped applications. Each chapter starts with a typical automation problem which is then approached using Test Studio specialized automation features. You will learn how to create record and playback functional, performance, and load tests. Furthermore, we will see how to insert verification steps, logical constructs, convenient logging operations, and how to convert test scripts in order to implement keyword and data-driven architectures. To endow your tests with additional flexibility, each recorded automation feature will be approached from its coded perspective through the usage of the underlying ArtOfTest Test Studio automation library. This book also illustrates how Test Studio can automate pre-conditions, test result inputting, and the capturing of system states during manual test case execution in order to keep the tester's attention focused on the important details.

Who is this book for?

If you are a beginner or a professional QA who is seeking a fast, clear, and direct to the point start in automated software testing inside Test Studio, this book is for you. You should be familiar with the .NET framework, mainly Visual Studio, C#, and SQL, as the book's examples rely on them. Prior testing knowledge will also be helpful.

What you will learn

Create functional automated tests using the recorder

Add logic to the recorded tests

Insert proper verification, and wait and log steps

Customize tests in code

Produce Keyword tests

Generate data-driven tests by connecting tests to various data sources

Export, create and run manual tests

Create and execute test lists

Gather performance test metrics

Carry out performance and load testing while collecting necessary metrics

What do you get with Print?

Instant access to your digital copy whilst your Print order is Shipped

Paperback book shipped to your preferred address

Redeem a companion digital copy on all Print orders

Access this title in our online reader with advanced features

DRM FREE - Read whenever, wherever and however you want

Frequently bought together

$54.99

Learning Software Testing with Test Studio

$60.99

$48.99

Total $ 164.97

JamesR Mar 24, 2014

This is really helpful book. You know the books which describe you solution but nothing mention about the problem. This book is problem driven and explain the solutions in details and also something which I liked really much is that each chapter present a situation (from the real world) and after that explains the problem followed by the solution.I found really interesting the chapter related to Mobile Testing which cover the problems from deployment of application under test to generating test report. Some of the chapters illustrate how to use testing techniques like decision tables and state transition diagrams using Test Studio IDE which is very cool because to know the particular technique is valuable, but to know how to apply it is priceless. You definitely have to find a way to read this book because it demonstrate best practices in functional automation and manual testing as well as performance testing and it can become your favorite book which have answer for most of the questions you asked yourself every day.This book contains examples with Test Studio IDE as well as theoretical information which I found very useful and up to date.

Amazon Verified review

W Boudville Mar 11, 2014

Software testing is one of these things that most programmers acknowledge as a necessary evil. Few programmers, and perhaps especially the new ones, regard this as interesting or challenging. But if you have been programming for some time, you should be aware of how important it is to test. And to build testing in as an inherent part of the development process. To this ends, Test Studio offers a framework to minimise some of the manual steps. Chapter 1 explains how Test Studio is compatible with data driven architecture and functional test automation. But it can also enhance manual testing.However, the main thrust of the rest of the book is to go from manual testing to the automating of it. You can see how unit tests can be simply hooked up in Test Studio. The framework is like a debugger, but at a higher level. In similarity with a debugger, you can define what happens when a test breaks - does the framework continue or abort. How serious is a test failure? This is important, for it recognises that some unit tests could fail in benign ways. That a fail has a gradation of meanings.For manual testing, some people have written tests in Microsoft Excel workbooks. Test Studio can import these very easily. Or, if you define tests in Test Studio, there is the means to export to Excel format.There is also comprehensive reporting ability. And another type of testing. For performance. Where you want to measure certain quantities during a test run, or during a set of runs. The most common is simply the run time or the wall clock time. Care has also been taken to handle the important case of web server performance. Perhaps germane to your situation.

NazHus Jan 30, 2014

Learning Software Testing with Test Studio teaches you how to reliably automate test cases. It takes a less-theory and more hands-on approach, (which i liked)Furthermore, you learn logical constructs, and convenient logging operations, as well as converting test scripts in order to implement keyword and data-driven architectures.Overall the book keeps the focus on the important details. A reference worth keeping.

John Mar 17, 2015

This book came in handy when our company decided to run with Test Studio as our automation tool. While Test Studio is pretty straight-forward and a very easy to use, there are some instances where their very detailed documentation and forums were not able to answer some of my questions. This book was able to answer three issues that I was not able to get satisfactory answers to. Since these issues were addressed as direct examples in this book, I'm assuming it must have been that the author came across the same questions.Well written with a good flow and each example builds upon the previous ones. This can be used as an easy reference once you have gone through it and put it up on your bookshelf.I gave it 4 out of 5 only because the book is not updated and the screenshots are from an earlier version of Test Studio. Test Studio is current on release 14 and the book looks like it is using version 12 or earlier.

Mikel Viera Apr 03, 2014

helps you get started with Test Studio to design your automated solution and tests. you can reliably automate test cases This book is all about less theory and more hands-on examples to present a complete manual and automated solution for your ASP .NET, WPF, Silverlight or iOS apps. Each chapter starts with a typical automation problem which is then approached using Test Studio specialized automation features.

Learning Software Testing with Test Studio: Embark on the exciting journey of test automation, execution, and reporting in Test Studio with this practical tutorial with this book and ebook

What do you get with Print?

Learning Software Testing with Test Studio

Chapter 1. Introduction

Testing concepts

Test case

Executing a test case

The exit criteria

The pesticide paradox

Element recognition

Testing phases

Testing techniques

Test Studio uncovered through the chapters

Approach

Setting up your environment

The File Comparer application

Reports

Additional files

Other reference sources

Page 1 of 8

Key benefits

Description

Who is this book for?

What you will learn

Product Details

What do you get with Print?

Product Details

Frequently bought together

Table of Contents

Recommendations for you

Customer reviews

People who bought this also bought

About the author

FAQs

Learning Software Testing with Test Studio: Embark on the exciting journey of test automation, execution, and reporting in Test Studio with this practical tutorial with this book and ebook

What do you get with Print?

Contact Details

Shipping Address

Billing Address

Key benefits

Description

Who is this book for?

What you will learn

Product Details

What do you get with Print?

Contact Details

Shipping Address

Billing Address

Product Details

Packt Subscriptions

Frequently bought together

Table of Contents

Recommendations for you

Customer reviews

People who bought this also bought

About the author

FAQs