Scaling up with Docker and Grid

Introduction

In previous sections, we have learnt how to run tests in parallel on the Cucumber level locally.  This is all well and good when you have a small number of tests, but let’s imagine we have a large number of tests / feature files that we want to run in parallel, this is where we would need to start thinking about strategies on how to scale up our parallel test execution.

A popular solution for this is to create a Selenium Grid within Docker containers to run tests in parallel across different browsers (nodes) in the grid.

Let’s get started 😀

Parallelisation via TestNG Test Suites

When using TestNG, each test suite can be represented by a xml file. If you’ve been following this blog series from the beginning, you should already have a testng.xml file. Let’s now extend upon this. We will have a separate XML Test Suite file for each of our feature files.

Test Runners

We will now make a test runner class for each feature file we have.

BaseTestRunner

  1. Right-click the ‘src/test/java’ directory and select ‘New’ -> ‘Java Class’
    • Name the class ‘BaseTestRunner’ and click OK
  2. Add the following in the BaseTestRunner class…

SearchTestRunner

  1. Right-click the ‘src/test/java’ directory and select ‘New’ -> ‘Java Class’
    • Name the class ‘SearchTestRunner’ and click OK
  2. Add the following in the SearchTestRunner class…

TestNG XML Test Suites

base-suite.xml

  1. In IntelliJ, right-click the src/test directory and select ‘New’ -> ‘File’
    • Name the file ‘base-suite.xml’ and click OK
  2. Add the following in the ‘base-suite.xml’ file to make it call the ‘BaseTestRunner’ class…

search-suite.xml

  1. In IntelliJ, right-click the src/test directory and select ‘New’ -> ‘File’
    • Name the file ‘search-suite.xml’ and click OK
  2. Add the following in the ‘search-suite.xml’ file to make it call the ‘SearchTestRunner’ class…

Docker

Setting up Docker

Docker is a set of platform-as-a-service products that use OS-level virtualisation to deliver software in packages called containers. Containers are isolated from one another and bundle their own software, libraries and configuration files; they can communicate with each other through well-defined channels.

  1. Go to https://www.docker.com/products/docker-desktop and click the button to download Docker Desktop
  2. If this is your first time using Docker, you will be asked to login, click “Signup” to create a new account and follow the simple instructions on their site
  3. Once you’ve created your account, login and download Docker Desktop for your appropriate Operating System (e.g. Mac and Windows)
  4. Follow the normal process to install Docker Desktop on your Operating System
  5. Run Docker Desktop for the first time and work your way through the first time setup flow, which will be similar to below…
    • Click ‘Next’ on the “Welcome to Docker Desktop!” dialog
    • Click ‘OK’ to give privileged access to Docker Desktop (entering your password if necessary)
  6. Now wait for Docker to finish starting up and give confirmation that it is now up and running
  7. Once Docker is up and running, you should see the icon (similar to a whale) in your taskbar, right-click the icon and login to your Docker account

Docker-Compose

Docker-Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure all of your application’s services. Then, with a single command (docker-compose up), you create and start all the services from your configuration. It is just as easy to tear it all down too (docker-compose down).

Creating the Docker-Compose file

  1. With the project open in IntelliJ, right-click the project root in the Project explorer view and select ‘New’ -> ‘File’
    • Name the file ‘docker-compose.yaml’ and click OK
  2. Add the following in the docker-compose.yaml file (comments have been included to help explain what is going on)…

Adding the RemoteWebDrivers

Creating the ‘hubUrl’ constant

Before we create our remote web drivers that our dockerised grid will use, we need to specify the url of the hub, we will do this by adding a constant which reads from the $HUB_URL environment variable that we have in our docker-compose.yaml file

  1. Open up the ‘Settings’ class in the utils.selenium package and add the following constant…

    The whole thing should look similar to below…

Creating the ChromeRemoteWebDriver class

  1. Right-click on the utils.drivers class and select ‘New’ -> ‘Java Class’
    • Name the class ‘ChromeRemoteWebDriver’ and click OK
  2.  Make the class look like below…

Creating the FirefoxRemoteWebDriver class

  1. Right-click on the utils.drivers class and select ‘New’ -> ‘Java Class’
    • Name the class ‘ChromeRemoteWebDriver’ and click OK
  2.  Make the class look like below…

Refactoring start() methods in DriverController class

In out Docker-Compose file, you may notice we added an environment variable called ‘LOCAL’ which we set to have a value of false, this is so our DriverController can load the RemoteWebDriver if LOCAL=false, else it will load the normal local WebDriver instance.

Refactoring the startChrome() method

  1. Open up the ‘DriverController’ class in the ‘utils.selenium’ package and refactor the startChrome() method to look like below…

Refactoring the startFirefox() method

  1. Open up the ‘DriverController’ class in the ‘utils.selenium’ package and refactor the startChrome() method to look like below…

    The whole class should look like below with the new imports etc. added…
  2. Next, go in to your ‘CucumberHooks’ class in the ‘utils.hooks’ package and make sure all the hook methods throw the necessary exceptions, like below…

Dockerfile

Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image. Using docker build, users can create an automated build that executes several command-line instructions in succession.

To make it easier when creating the Dockerfile, you can use pre-existing images shared with the community on Docker Hub.

Docker Hub

Docker Hub is a service provided by Docker for finding and sharing container images with your team. It provides the following major features: Repositories: Push and pull container images. … Builds: Automatically build container images from GitHub and Bitbucket and push them to Docker Hub.

Creating the Dockerfile

  1. With the project open in IntelliJ, right-click the project root in the Project explorer view and select ‘New’ -> ‘File’
    • Name the file ‘Dockerfile’ and click OK
  2. Add the following into the Dockerfile (comments have been included above each line to help explain what we are doing)…

Building and Pushing your Docker Image

  1. Open up a Terminal / Command Prompt and change directory (cd) in to your project root…
    cd path/To/ProjectRoot
  2. Once in the project root, let’s build your image (changing <username> to your Docker account’s username)…
    docker build -t <username>/selenium-docker .
  3. Once your image is built, push the image to your Docker hub repository (again changing <username> to your Docker account’s username and also entering Docker login credentials if asked for)…
    docker push <username>/selenium-docker

Running our tests in Docker

Finally put the @Chrome tag above one feature file and the @Firefox one above the other for testing purposes.

We can now run our tests in parallel in the dockerised grid by starting up our containers with the following command…

We can just as easily tear it down with the command below…

Congratulations, in the next section we will start looking at how we can run all this in CI/CD with Jenkins 😀

Liked it? Take a second to support Thomas on Patreon!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.