Locating WebElements using WebDriver

Let's start this section by automating the Search feature from the Homepage of the demo application, http://demo-store.seleniumacademy.com/, which involves navigating to the homepage, typing the search text in the textbox, and executing the search. The code is as follows:

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;

import static org.assertj.core.api.AssertionsForClassTypes.assertThat;

public class SearchTest {

    WebDriver driver;

    @BeforeMethod
    public void setup() {
        System.setProperty("webdriver.chrome.driver",
                "./src/test/resources/drivers/chromedriver");
        driver = new ChromeDriver();
        driver.get("http://demo-store.seleniumacademy.com/");
    }

    @Test
    public void searchProduct() {
        // find search box and enter search string
        WebElement searchBox = driver.findElement(By.name("q"));
        searchBox.sendKeys("Phones");
        WebElement searchButton = 
                driver.findElement(By.className("search-button"));
        searchButton.click();
        assertThat(driver.getTitle())
                .isEqualTo("Search results for: 'Phones'");
    }

    @AfterMethod
    public void tearDown() {
        driver.quit();
    }
}

As you can see, there are three new things that are highlighted, as follows:

WebElement searchBox = driver.findElement(By.name("q"));

They are the findElement() method, the By.name() method, and the WebElement interface. The findElement() and By() methods instruct WebDriver to locate a WebElement on a web page, and once found, the findElement() method returns the WebElement instance of that element. Actions, such as click and type, are performed on a returned WebElement using the methods declared in the WebElement interface, which will be discussed in detail in the next section.

The findElement method

In UI automation, locating an element is the first step before executing any user actions on it. WebDriver's findElement() method is a convenient way to locate an element on the web page. According to WebDriver's Javadoc (http://selenium.googlecode.com/git/docs/api/java/index.html), the method declaration is as follows:

WebElement findElement(By by)

So, the input parameter for the findElement() method is the By instance. The By instance is a WebElement-locating mechanism. There are eight different ways to locate a WebElement on a web page. We will see each of these eight methods later in the chapter.

The return type of the findElement() method is the WebElement instance that represents the actual HTML element or component of the web page. The method returns the first WebElement that the driver comes across that satisfies the locating-mechanism condition. This WebElement instance will act as a handle to that component from then on. Appropriate actions can be taken on that component by the test-script developer using this returned WebElement instance.

If WebDriver doesn't find the element, it throws a runtime exception named NoSuchElementException, which the invoking class or method should handle.

The findElements method

For finding multiple elements matching the same locator criteria on a web page, the findElements() method can be used. It returns a list of WebElements found for a given locating mechanism. The method declaration of the findElements() method is as follows:

java.util.List findElements(By by)

The input parameter is the same as the findElement() method, which is an instance of the By class. The difference lies in the return type. Here, if no element is found, an empty list is returned and if there are multiple WebElements present that satisfy the locating mechanism, all of them are returned to the caller in a list.

Inspecting Elements with Developer Tools

Before we start exploring how to find elements on a page and what locator mechanism to use, we need to look at the HTML code of the page to understand the Document Object Model (DOM) tree, what properties or attributes are defined for the elements displayed on the page, and how JavaScript or AJAX calls are made from the application. browsers use the HTML code written for the page to render visual elements in the browser window. It uses other resources, including JavaScript, CSS, and images, to decide on the look, feel, and behavior of these elements.

Here is an example of a login page of the demo application and the HTML code written to render this page in a browser, as displayed in the following screenshot:

We need tools that can display the HTML code of the page in a structured and easy-to-understand format. Almost all browsers now offer Developer tools to inspect the structure of the page and associated resources.

Inspecting pages and elements with Mozilla Firefox

The newer versions of Mozilla Firefox provide built-in ways to inspect the page and elements. To inspect an element from the page, move the mouse over the desired element and right-click to open the pop-up menu. Select the Inspect Element option, as shown in the following screenshot:

This will display the Inspector tab with the HTML code in a tree format with the selected element highlighted, as shown in the following screenshot:

Using Inspector, we can also validate the XPath or CSS Selectors using the search box shown in the Inspector section. Just enter the XPath or CSS Selector and Inspector will highlight the elements that match the expression, as shown in the following screenshot:

The Developer tools provide various other debugging features. It also generates XPath and CSS selectors for elements. For this, select the desired element in the tree, right-click, and select the Copy > XPath or Copy > CSS Path option from the pop-up menu, as shown in the following screenshot:

This will paste the suggested XPath or CSS selector value to the clipboard to be used later with the findElement() method.

Inspecting pages and elements in Google Chrome with Developer Tools

Similar to Mozilla Firefox, Google Chrome also provides a built-in feature to inspect pages and elements. We can move the mouse over a desired element on the page, right-click to open the pop-up menu, and then select the Inspect element option. This will open Developer tools in the browser, which displays information similar to that of Firefox, as shown in the following screenshot:

Similar to Firefox, we can also test XPath and CSS Selectors in Google Chrome Developer tools. Press Ctrl + F (on Mac, use Command + F) in the Elements tab. This will display a search box. Just enter XPath or CSS Selector, and matching elements will be highlighted in the tree, as shown in the following screenshot:

Chrome Developer Tools also provides a feature where you can get the XPath for an element by right-clicking on the desired element in the tree and selecting the Copy XPath option from the pop-up menu.

Similar to Mozilla Firefox and Google Chrome, you will find similar Developer tools in any major browser, including Microsoft Internet Explorer and Edge.

Browser developer tools come in really handy during the test-script development. These tools will help you to find the locator details for the elements with which you need to interact as part of the test. These tools parse the code for a page and display the information in a hierarchal tree.

WebElements on a web page may not have all the attributes declared. It is up to the developer of the test script to select the attribute that uniquely identifies the WebElement on the web page for the automation.

Using the By locating mechanism

By is the locating mechanism passed to the findElement() method or the findElements() method to fetch the respective WebElement(s) on a web page. There are eight different locating mechanisms; that is, eight different ways to identify

an HTML element on a web page. They are located by ID, Name, ClassName, TagName, LinkText, PartialLinkText, XPath, and CSS Selector.

The By.id() method

On a web page, each element is uniquely identified by an ID attribute, which is optionally provided. An ID can be assigned manually by the developer of the web application or left to be dynamically generated by the application. Dynamically-generated IDs can be changed on every page refresh or over a period of time. Now, consider the HTML code of the Search box:

<input id="search" type="search" name="q" value="" class="input-text required-entry" maxlength="128" placeholder="Search entire store here..." autocomplete="off">

In the preceding code, the id attribute value of the search box is search.

Let's see how to use the ID attribute as a locating mechanism to find the Search box:

@Test
public void byIdLocatorExample() {
    WebElement searchBox = driver.findElement(By.id("search"));
    searchBox.sendKeys("Bags");
    searchBox.submit();
    assertThat(driver.getTitle())
            .isEqualTo("Search results for: 'Bags'");
}

In preceding code, we used the By.id() method and the search box's id attribute value to find the element.

Here, try to use the By.id identifier, and use the name value (that is, q) instead of the id value (that is, search). Modify line three as follows:

WebElement searchBox = driver.findElement(By.id("q")); 

The test script will fail to throw an exception, as follows:

Exception in thread "main" org.openqa.selenium.NoSuchElementException: Unable to locate element: {"method":"id","selector":"q"}

WebDriver couldn't find an element by id whose value is q. Thus, it throws an exception saying NoSuchElementException.

The By.name() method

As seen earlier, every element on a web page has many attributes. Name is one of them. For instance, the HTML code for the Search box is:

<input id="search" type="search" name="q" value="" class="input-text required-entry" maxlength="128" placeholder="Search entire store here..." autocomplete="off">

Here, name is one of the many attributes of the search box, and its value is q. If we want to identify this search box and set a value in it in your test script, the code will look as follows:

@Test
public void searchProduct() {
    // find search box and enter search string
    WebElement searchBox = driver.findElement(By.name("q"));
    searchBox.sendKeys("Phones");
    searchBox.submit();
    assertThat(driver.getTitle())
            .isEqualTo("Search results for: 'Phones'");
}

If you observe line four, the locating mechanism used here is By.name and the name is q. So, where did we get this name from? As discussed in the previous section, it is the browser developer tools that helped us get the name of the button. Launch Developer tools and use the inspect elements widget to get the attributes of an element.

The By.className() method

Before we discuss the className() method, we have to talk a little about style and CSS. Every HTML element on a web page, generally, is styled by the web page developer or designer. It is not mandatory that each element should be styled, but they generally are to make the page appealing to the end user.

So, in order to apply styles to an element, they can be declared directly in the element tag, or placed in a separate file called the CSS file and can be referenced in the element using the class attribute. For instance, a style attribute for a button can be declared in a CSS file as follows:

.buttonStyle{
    width: 50px;
    height: 50px;
    border-radius: 50%;
    margin: 0% 2%;
}

Now, this style can be applied to the button element in a web page as follows:

<button name="sampleBtnName" id="sampleBtnId" class="buttonStyle">I'm Button</button>

So, buttonStyle is used as the value for the class attribute of the button element, and it inherits all the styles declared in the CSS file. Now, let's try this on our Homepage. We will try to make WebDriver identify the search button using its class name and click on it.

First, in order to get the class name of the search button, as we know, we will use Developers tools to fetch it. After getting it, change the location mechanism to By.className and specify the class attribute value in it. The code for that is as follows:

@Test
public void byClassNameLocatorExample() {
    WebElement searchBox = driver.findElement(By.id("search"));
    searchBox.sendKeys("Electronics");
    WebElement searchButton = 
            driver.findElement(By.className("search-button"));
    searchButton.click();
    assertThat(driver.getTitle())
            .isEqualTo("Search results for: 'Electronics'");
}

In the preceding code, we have used the By.className locating mechanism by passing the class attribute value to it.

Sometimes, an element might have multiple values given for the class attribute. For example, the Search button has button and search-button values specified in the class attribute in the following HTML snippet:

<button type="submit" title="Search" class="button search-button"><span><span>Search</span></span></button>

We have to use one of the values of the class attribute with the By.className method. In this case, we can either use button or search-button, whichever uniquely identifies the element.

The By.linkText() method

As the name suggests, the By.linkText locating mechanism can only be used to identify the HTML links. Before we start discussing how WebDriver can be commanded to identify a link element using link text, let's see what an HTML link element looks like. The HTML link elements are represented on a web page using the <a> tag, an abbreviation for the anchor tag. A typical anchor tag looks like this:

<a href="http://demo-store.seleniumacademy.com/customer/account/" title="My Account">My Account</a>

Here, href is the link to a different page where your web browser will take you when you click on the link. So, the preceding HTML code when rendered by the browser looks like this:

This MY ACCOUNT is the link text. So the By.linkText locating mechanism uses this text on an anchor tag to identify the WebElement. The code would look like this:

@Test
public void byLinkTextLocatorExample() {
    WebElement myAccountLink = 
            driver.findElement(By.linkText("MY ACCOUNT"));
    myAccountLink.click();
    assertThat(driver.getTitle())
            .isEqualTo("Customer Login");
}

Here, the By.linkText locating mechanism is used to identify the MY ACCOUNT link.

The linkText and partialLinkText methods are case-sensitive.

The By.partialLinkText() method

The By.partialLinkText locating mechanism is an extension of the By.linkText locator. If you are not sure of the entire link text or want to use only part of the link text, you can use this locator to identify the link element. So, let's modify the previous example to use only partial text on the link; in this case, we will use Privacy from the Privacy Policy link in the site footer:

The code would look like this:

@Test
public void byPartialLinkTextLocatorExample() {
    WebElement orderAndReturns = 
            driver.findElement(By.partialLinkText("PRIVACY"));
    orderAndReturns.click();
    assertThat(driver.getTitle())
            .isEqualTo("Privacy Policy");
}

What happens if there are multiple links whose text has Privacy in it? That is a question for the findElement() method rather than the locator. Remember when we discussed the findElement() method earlier, it will return only the first WebElement that it comes across. If you want all the WebElements that contain Privacy in its link text, use the findElements() method, which will return a list of all those elements.

Use WebDriver's findElements() method if you think you need all the WebElements that satisfy a locating-mechanism condition.

The By.tagName() method

Locating an element by tag name is slightly different from the locating mechanisms we saw earlier. For example, on a Homepage, if you search for an element with the button tag name, it will result in multiple WebElements because there are nine buttons present on the Homepage. So, it is always advisable to use the findElements() method rather than the findElement() method when trying to locate elements using tag names.

Let's see how the code looks when a search for the number of links present on a Homepage is made:

@Test
public void byTagNameLocatorExample() {

    // get all links from the Home page
    List<WebElement> links = driver.findElements(By.tagName("a"));

    System.out.println("Found links:" + links.size());

    // print links which have text using Java 8 Streams API
    links.stream()
            .filter(elem -> elem.getText().length() > 0)
            .forEach(elem -> System.out.println(elem.getText()));
}

In the preceding code, we have used the By.tagName locating mechanism and the findElements() method, which return a list of all the links, that is, the a anchor tags defined on the page. On line five, we printed the size of the list, and then printed text of only links where the text has been provided. We use the Java 8 Stream API to filter the element list and output the text value by calling the getText() method. This will generate the following output:

Found links:88
 ACCOUNT
 CART
 WOMEN
 ...

The By.xpath() method

WebDriver uses XPath to identify a WebElement on the web page. Before we see how it does that, let's quickly look at the syntax for XPath. XPath is a short name for the XML path, the query language used for searching XML documents. The HTML for our web page is also one form of the XML document. So, in order to identify an element on an HTML page, we need to use a specific XPath syntax:

• The root element is identified as //.
• To identify all the div elements, the syntax will be //div.
• To identify the link tags that are within the div element, the syntax will be //div/a.
• To identify all the elements with a tag, we use *. The syntax will be //div/*.
• To identify all the div elements that are at three levels down from the root, we can use //*/*/div.
• To identify specific elements, we use attribute values of those elements, such as //*/div/a[@id='attrValue'], which will return the anchor element. This element is at the third level from the root within a div element and has an id value of attrValue.

So, we need to pass the XPath expression to the By.xpath locating mechanism to make it identify our target element.

Now, let's see the code example and how WebDriver uses this XPath to identify the element:

@Test
public void byXPathLocatorExample() {
    WebElement searchBox = 
            driver.findElement(By.xpath("//*[@id='search']"));
    searchBox.sendKeys("Bags");
    searchBox.submit();
    assertThat(driver.getTitle())
            .isEqualTo("Search results for: 'Bags'");
}

In the preceding code, we are using the By.xpath locating mechanism and passing the XPath of the WebElement to it.

One disadvantage of using XPath is that it is costly in terms of time. For every element to be identified, WebDriver actually scans through the entire page, which is very time consuming, and too much usage of XPath in your test script will actually make it too slow to execute.

The By.cssSelector() method

The By.cssSelector() method is similar to the By.xpath() method in its usage, but the difference is that it is slightly faster than the By.xpath locating mechanism. The following are the commonly used syntaxes to identify elements:

• To identify an element using the div element with the #flrs ID, we use the #flrs syntax
• To identify the child anchor element, we use the #flrs > a syntax, which will return the link element
• To identify the anchor element with its attribute, we use the #flrs > a[a[href="/intl/en/about.html"]] syntax

Let's try to modify the previous code, which uses the XPath locating mechanism to use the cssSelector mechanism:

@Test
public void byCssSelectorLocatorExample() {
    WebElement searchBox = 
            driver.findElement(By.cssSelector("#search"));
    searchBox.sendKeys("Bags");
    searchBox.submit();
    assertThat(driver.getTitle())
            .isEqualTo("Search results for: 'Bags'");
}

The preceding code uses the By.cssSelector locating mechanism, which uses the css selector ID of the Search box.

Let's look at a slightly complex example. We will try to identify the About Us on the Homepage:

@Test
public void byCssSelectorLocatorComplexExample() {

    WebElement aboutUs =
            driver.findElement(By
                    .cssSelector("a[href*='/about-magento-demo-store/']"));

    aboutUs.click();

    assertThat(driver.getTitle())
            .isEqualTo("About Us");
}

The preceding code uses the cssSelector() method to find the anchor element identified by its href attribute.