Scaling up with Docker and Grid

Introduction

In previous sections, we have learnt how to run tests in parallel on the Cucumber level locally. This is all well and good when you have a small number of tests, but let’s imagine we have a large number of tests / feature files that we want to run in parallel, this is where we would need to start thinking about strategies on how to scale up our parallel test execution.

A popular solution for this is to create a Selenium Grid within Docker containers to run tests in parallel across different browsers (nodes) in the grid.

Let’s get started 😀

Parallelisation via TestNG Test Suites

When using TestNG, each test suite can be represented by a xml file. If you’ve been following this blog series from the beginning, you should already have a testng.xml file. Let’s now extend upon this. We will have a separate XML Test Suite file for each of our feature files.

Because we will be wanting to rely on TestNG for parallelisation of our tests when scaling up with Docker and Selenium Grid, it is no longer a good idea to use Courgette-JVM. Previously, Courgette-JVM automated the process of creating test runners on the fly for either each of our feature files, or each of our scenarios even if that is what we configured. This is all good, but if we wanted greater control over the Test Runners and which test suites call which test runners etc, we will want to remove this and add back in the Cucumber and TestNG dependencies in our build.gradle file, in part due to Courgette-JVM including old versions of the dependencies we want to add back in.

Removing Courgette-JVM

First of all, right-click the existing TestRunner class and select ‘Delete…’ -> ‘OK’

Next up, open up the build.gradle file and remove the runTests() task…

task runTests(type: Test) {
    useTestNG() // specify that we use TestNG instead of JUnit
    include '**/TestRunner.class'
    outputs.upToDateWhen { false }
}

Finally, remove the Courgette-JVM dependency line…

testCompile group: 'io.github.prashant-ramcharan', name: 'courgette-jvm', version: '3.0.1'

Adding back the Cucumber and TestNG dependencies

A lot has changed with the new version 4 of Cucumber, however it is still awesome so let’s use it 🙂

Add the following dependencies in your build.gradle file…

compile group: 'io.cucumber', name: 'cucumber-testng', version: '4.8.0'
compile group: 'io.cucumber', name: 'cucumber-java', version: '4.8.0'
compile group: 'io.cucumber', name: 'cucumber-core', version: '4.8.0'

The whole thing should look like below…

group = 'com.testifyqagradleframework'
version = '1.0-SNAPSHOT'

apply plugin: 'java'
apply plugin: 'application'
apply plugin: 'com.github.johnrengelman.shadow'

compileJava {
    sourceCompatibility = 1.8
    targetCompatibility = 1.8
}

repositories {
    mavenCentral()
    jcenter()
}

buildscript {
    repositories {
        mavenCentral()
        jcenter()
    }
    dependencies {
        classpath 'com.github.jengelman.gradle.plugins:shadow:5.1.0'
    }
}

dependencies {
    compile group: 'org.seleniumhq.selenium', name: 'selenium-java', version: '3.141.59'
    compile group: 'org.apache.logging.log4j', name: 'log4j-core', version:'2.11.1'
    compile group: 'org.apache.logging.log4j', name: 'log4j-api', version:'2.11.1'
    compile group: 'com.google.guava', name: 'guava', version:'25.1-jre'
    compile group: 'com.google.code.gson', name: 'gson', version:'2.8.5'
    compile group: 'io.github.bonigarcia', name: 'webdrivermanager', version: '3.3.0'
    compile group: 'org.json', name: 'json', version:'20180130'
    compile group: 'io.cucumber', name: 'cucumber-testng', version: '4.8.0'
    compile group: 'io.cucumber', name: 'cucumber-java', version: '4.8.0'
    compile group: 'io.cucumber', name: 'cucumber-core', version: '4.8.0'
}

shadowJar {
    mainClassName = 'seleniumgradle'
    baseName = 'seleniumgradle-test'
    classifier = 'tests'
    from sourceSets.test.output
    configurations = [project.configurations.testRuntime]
}

tasks.withType(Test) {
    systemProperties = System.getProperties()
    systemProperties.remove("java.endorsed.dirs")
}

Test Runners

We will now make a test runner class for each feature file we have.

BaseTestRunner

Right-click the ‘src/test/java’ directory and select ‘New’ -> ‘Java Class’
- Name the class ‘BaseTestRunner’ and click OK

Add the following in the BaseTestRunner class…

import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;

@CucumberOptions(
        features = "src/test/resources/features/BaseScenarios.feature",
        glue = {"utils.hooks", "steps"},
        tags = {"~@Ignore"},
        plugin = {"html:target/cucumber-reports/cucumber-pretty",
                "pretty",
                "html:target/cucumber-reports/cucumber-pretty",
                "json:target/cucumber-reports/CucumberTestReport.json",
                "rerun:target/cucumber-reports/rerun.txt"
        })
public class BaseTestRunner extends AbstractTestNGCucumberTests {
}

SearchTestRunner

Right-click the ‘src/test/java’ directory and select ‘New’ -> ‘Java Class’
- Name the class ‘SearchTestRunner’ and click OK

Add the following in the SearchTestRunner class…

import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;

@CucumberOptions(
        features = "src/test/resources/features/SearchScenarios.feature",
        glue = {"utils.hooks", "steps"},
        tags = {"~@Ignore"},
        plugin = {"html:target/cucumber-reports/cucumber-pretty",
                "pretty",
                "html:target/cucumber-reports/cucumber-pretty",
                "json:target/cucumber-reports/CucumberTestReport.json",
                "rerun:target/cucumber-reports/rerun.txt"
        })
public class SearchTestRunner extends AbstractTestNGCucumberTests {
}

TestNG XML Test Suites

base-suite.xml

In IntelliJ, right-click the src/test directory and select ‘New’ -> ‘File’
- Name the file ‘base-suite.xml’ and click OK

Add the following in the ‘base-suite.xml’ file to make it call the ‘BaseTestRunner’ class…

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Product Test Suite" verbose="1" parallel="tests" thread-count="1" configfailurepolicy="continue">
    <test name="Product Acceptance Tests" annotations="JDK" preserve-order="true">
        <classes>
            <class name="BaseTestRunner"/>
        </classes>
    </test>
</suite>

search-suite.xml

In IntelliJ, right-click the src/test directory and select ‘New’ -> ‘File’
- Name the file ‘search-suite.xml’ and click OK

Add the following in the ‘search-suite.xml’ file to make it call the ‘SearchTestRunner’ class…

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Product Test Suite" verbose="1" parallel="tests" thread-count="1" configfailurepolicy="continue">
    <test name="Product Acceptance Tests" annotations="JDK" preserve-order="true">
        <classes>
            <class name="SearchTestRunner"/>
        </classes>
    </test>
</suite>

Docker

Setting up Docker

Docker is a set of platform-as-a-service products that use OS-level virtualisation to deliver software in packages called containers. Containers are isolated from one another and bundle their own software, libraries and configuration files; they can communicate with each other through well-defined channels.

Go to https://www.docker.com/products/docker-desktop and click the button to download Docker Desktop
If this is your first time using Docker, you will be asked to login, click “Signup” to create a new account and follow the simple instructions on their site
Once you’ve created your account, login and download Docker Desktop for your appropriate Operating System (e.g. Mac and Windows)
Follow the normal process to install Docker Desktop on your Operating System
Run Docker Desktop for the first time and work your way through the first time setup flow, which will be similar to below…
- Click ‘Next’ on the “Welcome to Docker Desktop!” dialog
- Click ‘OK’ to give privileged access to Docker Desktop (entering your password if necessary)
Now wait for Docker to finish starting up and give confirmation that it is now up and running
Once Docker is up and running, you should see the icon (similar to a whale) in your taskbar, right-click the icon and login to your Docker account

Docker-Compose

Docker-Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure all of your application’s services. Then, with a single command (docker-compose up), you create and start all the services from your configuration. It is just as easy to tear it all down too (docker-compose down).

Creating the Docker-Compose file

With the project open in IntelliJ, right-click the project root in the Project explorer view and select ‘New’ -> ‘File’
- Name the file ‘docker-compose.yaml’ and click OK

Add the following in the docker-compose.yaml file (comments have been included to help explain what is going on)…

version: "3.1" #version of docker-compose reference to use
services: #begin listing our services we want to use for our Grid
  hub:
    image: selenium/hub #get the selenium grid image
    ports:
      - "4444:4444" #set the ports of the hub to 4444:4444
  chrome:
    image: selenium/node-chrome #get the chrome node image
    links:
      - hub #chrome node image requires the hub to work
    environment:
      - HUB_PORT_4444_TCP_ADDR=hub #set the hub port for the chrome node
      - HUB_PORT_4444_TCP_PORT=4444
  firefox:
    image: selenium/node-firefox #get the firefox node image
    links:
      - hub #firefox node image requires the hub to work
    environment:
      - HUB_PORT_4444_TCP_ADDR=hub #set the hub port for the firefox node
      - HUB_PORT_4444_TCP_PORT=4444
  base-tests:
    image: username/selenium-docker #change username to your docker account's username
    container_name: base-tests
    depends_on: #this test image depends on the grid being set up
      - firefox
      - chrome
    environment:
      - TEST_SUITE=base-suite.xml #run the base-suite.xml test suite
      - HUB_URL=hub
      - LOCAL=false
  search-tests:
    image: username/selenium-docker #change username to your docker account's username
    container_name: search-tests
    depends_on: #this test image depends on the grid being set up
      - firefox
      - chrome
    environment:
      - TEST_SUITE=search-suite.xml #run the search-suite.xml test suite
      - HUB_URL=hub
      - LOCAL=false

Adding the RemoteWebDrivers

Creating the ‘hubUrl’ constant

Before we create our remote web drivers that our dockerised grid will use, we need to specify the url of the hub, we will do this by adding a constant which reads from the $HUB_URL environment variable that we have in our docker-compose.yaml file

Open up the ‘Settings’ class in the utils.selenium package and add the following constant…

public static String hubUrl = System.getenv("HUB_URL");

The whole thing should look similar to below…

package utils.selenium;

public class Settings {

    public static String baseUrl = "https://start.duckduckgo.com/";
    public static String weHighlightedColour = "arguments[0].style.border='5px solid blue'";
    public static String wdHighlightedColour = "arguments[0].style.border='5px solid green";
    public static String hubUrl = System.getenv("HUB_URL");
}

Creating the ChromeRemoteWebDriver class

Right-click on the utils.drivers class and select ‘New’ -> ‘Java Class’
- Name the class ‘ChromeRemoteWebDriver’ and click OK

Make the class look like below…

package utils.drivers;

import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.remote.RemoteWebDriver;

import java.net.URL;

import static utils.selenium.Settings.hubUrl;

public class ChromeRemoteWebDriver {

    public static RemoteWebDriver loadChromeDriver(String chromeArgument) throws Exception{
        ChromeOptions options = new ChromeOptions();
        options.addArguments(chromeArgument);
        return new RemoteWebDriver(new URL("http://"+hubUrl+":4444/wd/hub"), options);
    }
}

Creating the FirefoxRemoteWebDriver class

Right-click on the utils.drivers class and select ‘New’ -> ‘Java Class’
- Name the class ‘ChromeRemoteWebDriver’ and click OK

Make the class look like below…

package utils.drivers;

import org.openqa.selenium.firefox.FirefoxOptions;
import org.openqa.selenium.remote.RemoteWebDriver;

import java.net.URL;

import static utils.selenium.Settings.hubUrl;

public class FirefoxRemoteWebDriver {

    public static RemoteWebDriver loadFirefoxDriver(String firefoxArgument) throws Exception {
        FirefoxOptions options = new FirefoxOptions();
        options.addArguments(firefoxArgument);
        return new RemoteWebDriver(new URL("http://"+hubUrl+":4444/wd/hub"), options);
    }
}

How to be able to still run tests locally

Because we now have some environment variables such as “LOCAL”, if we try to run a feature file locally (e.g. using the @Web) tag, it will fail with a NullPointerException due to it not being able to find the “LOCAL” environment variable. To fix this, we should add a dependency called DotEnv to our build.gradle file, which will allow us to list environment variables in a .env file and have it read from there 🙂

Add the following dependency within the dependencies braces of the build.gradle file …
```
compile group: 'io.github.cdimascio', name: 'java-dotenv', version: '5.1.3'
```
Next within IntelliJ, right-click on the project root in the Project explorer and select ‘New’ -> ‘File’
1. Name the file ‘.env’ and press <Return>
In the ‘.env’ file, add the following environment variable…
```
LOCAL=true
```

Refactoring start() methods in DriverController class

In out Docker-Compose file, you may notice we added an environment variable called ‘LOCAL’ which we set to have a value of false, this is so our DriverController can load the RemoteWebDriver if LOCAL=false, else it will load the normal local WebDriver instance.

Refactoring the startChrome() method

Open up the ‘DriverController’ class in the ‘utils.selenium’ package and refactor the startChrome() method to look like below…

public void startChrome(String arg) throws Exception {
    if(instance.webDriver != null) return;
    if(System.getenv("LOCAL").trim().equals("false")) {
        instance.webDriver = ChromeRemoteWebDriver.loadChromeDriver(arg);
    }
    else {
        instance.webDriver = ChromeWebDriver.loadChromeDriver(arg);
    }
}

Refactoring the startFirefox() method

Open up the ‘DriverController’ class in the ‘utils.selenium’ package and refactor the startChrome() method to look like below…

public void startFirefox(String arg) throws Exception {
    if(instance.webDriver != null) return;
    if(System.getenv("LOCAL").trim().equals("false")) {
        instance.webDriver = FirefoxRemoteWebDriver.loadFirefoxDriver(arg);
    }
    else {
        instance.webDriver = FirefoxWebDriver.loadFirefoxDriver(arg);
    }
}

The whole class should look like below with the new imports etc. added…

package utils.selenium;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.openqa.selenium.WebDriver;
import utils.drivers.ChromeRemoteWebDriver;
import utils.drivers.ChromeWebDriver;
import utils.drivers.FirefoxRemoteWebDriver;
import utils.drivers.FirefoxWebDriver;

public class DriverController {

    public static DriverController instance = new DriverController();

    WebDriver webDriver;

    private static Logger log = LogManager.getLogger(DriverController.class.getName());

    public void startChrome(String arg) throws Exception {
        if(instance.webDriver != null) return;
        if(System.getenv("LOCAL").trim().equals("false")) {
            instance.webDriver = ChromeRemoteWebDriver.loadChromeDriver(arg);
        }
        else {
            instance.webDriver = ChromeWebDriver.loadChromeDriver(arg);
        }
    }

    public void startFirefox(String arg) throws Exception {
        if(instance.webDriver != null) return;
        if(System.getenv("LOCAL").trim().equals("false")) {
            instance.webDriver = FirefoxRemoteWebDriver.loadFirefoxDriver(arg);
        }
        else {
            instance.webDriver = FirefoxWebDriver.loadFirefoxDriver(arg);
        }
    }

    public void stopWebDriver() {
        if (instance.webDriver == null) return;

        try
        {
            instance.webDriver.quit();
        }
        catch (Exception e)
        {
            log.error(e + "::WebDriver stop error");
        }

        instance.webDriver = null;
        log.debug(":: WebDriver stopped");
    }
}

Next, go in to your ‘CucumberHooks’ class in the ‘utils.hooks’ package and make sure all the hook methods throw the necessary exceptions, like below…

package utils.hooks;

import io.cucumber.java.After;
import io.cucumber.java.Before;
import utils.selenium.DriverController;

import java.io.FileInputStream;
import java.util.Properties;

public class CucumberHooks {

    @Before("@Web")
    public void beforeWeb() throws Exception {
        Properties browserProps = new Properties();
        browserProps.load(new FileInputStream("src/test/resources/config.properties"));

        String browser = browserProps.getProperty("browserName");

        if (browser.equalsIgnoreCase("chrome")) {
            DriverController.instance.startChrome("--disable-extensions");
        }
        else if (browser.equalsIgnoreCase("firefox")) {
            DriverController.instance.startFirefox("--disable-extensions");
        }
    }

    @Before("@Chrome")
    public void beforeChrome() throws Exception {
        DriverController.instance.startChrome("--disable-extensions");
    }

    @Before("@Firefox")
    public void beforeFirefox() throws Exception {

        DriverController.instance.startFirefox("--disable-extensions");
    }

    @Before("@HeadlessChrome")
    public void beforeChromeHeadless() throws Exception {
        DriverController.instance.startChrome("--headless");
    }

    @Before("@HeadlessFirefox")
    public void beforeHeadlessFirefox() throws Exception {
        DriverController.instance.startFirefox("--headless");
    }

    @After
    public void stopWebDriver() {
        DriverController.instance.stopWebDriver();
    }
}

Dockerfile

A Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image. Using docker build, users can create an automated build that executes several command-line instructions in succession.

To make it easier when creating the Dockerfile, you can use pre-existing images shared with the community on Docker Hub.

Docker Hub

Docker Hub is a service provided by Docker for finding and sharing container images with your team. It provides the following major features: Repositories: Push and pull container images. … Builds: Automatically build container images from GitHub and Bitbucket and push them to Docker Hub.

Creating the Dockerfile

With the project open in IntelliJ, right-click the project root in the Project explorer view and select ‘New’ -> ‘File’
- Name the file ‘Dockerfile’ and click OK

Add the following into the Dockerfile (comments have been included above each line to help explain what we are doing)…

#Initialise a new build stage and sets the Base Image for subsequent instructions.
#As such, a valid Dockerfile must start with a FROM instruction. The image can be any valid image.
#Here we are getting an image running Linux Alpine OS which has Java JDK 8 on it
FROM openjdk:8-jdk-alpine

#Add a bash shell to the base image defined above
RUN apk add --no-cache bash

#create a directory called 'app'
RUN mkdir /app

#The WORKDIR instruction sets the working directory for any RUN, CMD, ENTRYPOINT, COPY and ADD instructions that follow it in the Dockerfile.
#Here we are setting the app directory we created above as the working directory
WORKDIR /app

#Copy all files and folders from our project root in to our 'app' working directory
COPY . /app

#Run the shadowJar gradle task using the bash shell we added previously, in order to package our JAR file
RUN ./gradlew shadowJar

#An ENTRYPOINT allows you to configure a container that will run as an executable.
#Here we are executing our JAR file via TestNG with a given xml test suite passed in from our $TEST_SUITE environment variable
ENTRYPOINT ["/bin/sh", "-c", "java -cp 'build/libs/seleniumgradle-test-1.0-SNAPSHOT-tests.jar' org.testng.TestNG $TEST_SUITE"]

Building and Pushing your Docker Image

Open up a Terminal / Command Prompt and change directory (cd) in to your project root…
```
cd path/To/ProjectRoot
```
Once in the project root, let’s build your image (changing <username> to your Docker account’s username)…
```
docker build -t <username>/selenium-docker .
```
Once your image is built, push the image to your Docker hub repository (again changing <username> to your Docker account’s username and also entering Docker login credentials if asked for)…
```
docker push <username>/selenium-docker
```

Tidying up our imports

Because we are now using a new version 4 of Cucumber, a lot of the libraries have moved around and old imports we are using have been deprecated, let’s tidy this up now…

CucumberHooks

Open up the ‘CucumberHooks’ class in the ‘utils.hooks’ package and swap out the Before/After imports to now be like below…

import io.cucumber.java.After;
import io.cucumber.java.Before;

The whole thing should look like below…

package utils.hooks;

import io.cucumber.java.After;
import io.cucumber.java.Before;
import utils.selenium.DriverController;

import java.io.FileInputStream;
import java.util.Properties;

public class CucumberHooks {

    @Before("@Web")
    public void beforeWeb() throws Exception {
        Properties browserProps = new Properties();
        browserProps.load(new FileInputStream("src/test/resources/config.properties"));

        String browser = browserProps.getProperty("browserName");

        if (browser.equalsIgnoreCase("chrome")) {
            DriverController.instance.startChrome("--disable-extensions");
        }
        else if (browser.equalsIgnoreCase("firefox")) {
            DriverController.instance.startFirefox("--disable-extensions");
        }
    }

    @Before("@Chrome")
    public void beforeChrome() throws Exception {
        DriverController.instance.startChrome("--disable-extensions");
    }

    @Before("@Firefox")
    public void beforeFirefox() throws Exception {
        DriverController.instance.startFirefox("--disable-extensions");
    }

    @Before("@HeadlessChrome")
    public void beforeChromeHeadless() throws Exception {
        DriverController.instance.startChrome("--headless");
    }

    @Before("@HeadlessFirefox")
    public void beforeHeadlessFirefox() throws Exception {
        DriverController.instance.startFirefox("--headless");
    }

    @After
    public void stopWebDriver() {
        DriverController.instance.stopWebDriver();
    }
}

Steps classes

For each step definition class that contains any step definitions, we will need to swap out the Given/When/Then imports to use the new ones like below…

import io.cucumber.java.en.Given;
import io.cucumber.java.en.When;
import io.cucumber.java.en.Then;
import io.cucumber.java.en.And;

Use your IDE to help you do this if needed.

Running our tests in Docker

Finally put the @Chrome tag above one feature file and the @Firefox one above the other for testing purposes.

We can now run our tests in parallel in the dockerised grid by starting up our containers with the following command…

docker-compose up

We can just as easily tear it down with the command below…

docker-compose down

Congratulations, in the next section we will start looking at how we can run all this in CI/CD with Jenkins 😀

Liked it? Take a second to support Thomas on Patreon!

A blog on QA Automation

Part 9a. Scaling up with Docker and Grid (Gradle)