Part 6. Parallel Test Execution (Protractor)

Parallel Test Execution

Introduction

Right now, when we run all of our tests in our framework, they are run sequentially (e.g. one at a time).  This is arguably ok for now as we only have a total of 4 test scenarios (2 scenarios per feature file).

But let’s imagine we have loads of tests in our framework, and those tests need testing across multiple browsers.  If we ran all these tests sequentially, the time it would take to complete would be relatively long.

This is where parallel test execution comes in. By using multiple threads, we can run our tests concurrently (at the same time) to reduce the total amount of time it takes to execute all the tests in the framework.

In today’s modern age, most computers have multiple processors, which we can use to our advantage to run tests in parallel locally. When we start to want to scale up even more, we can use Selenium Grid to run our test sessions on various nodes in parallel.

Thankfully, Protractor make it very easy to run our tests in parallel.

There are two different ways to run our tests in parallel.

The first way is by running multiple specs/feature files concurrently. For example, we can run our ‘find_a_restaurant.feature’ file and our ‘subscribe_to_newsletter.feature’ file simultaneously.

The second way is by running the same spec/feature file against multiple browsers in parallel.  For example, we could run our ‘find_a_restaurant.feature’ file against Chrome and Firefox simultaneously

In this blog part, we will set up our tests to be run in parallel both ways 🙂

Anyways, Let’s get started 😀

Running our tests concurrently

Running our feature files in parallel

  1. With the project open in Visual Studio Code, open up the ‘protractor.conf.js’ file and add the following capabilities in the ‘capabilities’ block, like below…
    capabilities: {
        browserName: 'chrome', // set test browser to 'chrome', can also be set to 'firefox' as well as some others
        shardTestFiles: true, // specs are sharded by file when set to true
        maxInstances: 2 // maximum number of browser instances that can run in parallel for this
    },

    The whole thing should look similar to below at this point…

    const
        { Photographer, TakePhotosOfFailures } = require('@serenity-js/protractor'),
        { SerenityBDDReporter } = require('@serenity-js/serenity-bdd'),
        { ConsoleReporter } = require('@serenity-js/console-reporter'),
        { ArtifactArchiver } = require('@serenity-js/core');
    
    exports.config = { // export the config so it is accessible by other files
        baseUrl: 'https://www.zagat.com', // // set the base url of our system under test
        directConnect: true, // your test script communicates directly Chrome Driver or Firefox Driver, bypassing any Selenium Server.
        allScriptsTimeout: 120000, // set global protractor timeout to 120 seconds (2 minutes)
        capabilities: {
            browserName: 'chrome', // set test browser to 'chrome', can also be set to 'firefox' as well as some others
            shardTestFiles: true, // specs are sharded by file when set to true
            maxInstances: 2 // maximum number of browser instances that can run in parallel for this
        },
        specs: ['./features/*.feature'], // set the location of our tests to any and all feature files in the 'features' folder
        framework: 'custom',
        frameworkPath: require.resolve('@serenity-js/protractor/adapter'), // this serenity-js module needs to be called as a custom framework
        onPrepare: () => {
            require('ts-node').register({ // register 'ts-node' to resolve all our package imports
                project: './tsconfig.json' // and handle TypeScript execution smoothly
            });
        },
        serenity: {
            //actors: new Actors(), // instantiate our Actors object so they can use their abilities
            runner: 'cucumber', // tell serenity-js that we want to use the Cucumber runner
            crew: [
                Photographer.whoWill(TakePhotosOfFailures), // or Photographer.whoWill(TakePhotosOfInteractions),
                new SerenityBDDReporter(),
                ConsoleReporter.forDarkTerminals(), // display execution summary in console
                ArtifactArchiver.storingArtifactsAt('./target/site/serenity') // store serenity reports at given location
            ]
        },
        cucumberOpts: {
            require: ['./features/**/*.ts', //require our step definition files
                      './support/**/*.ts'], // also require any necessary files in support folders and sub-folders (e.g. setup.ts)        
                      format: ['snippets:pending-steps.txt'], // workaround to get pending step definitions
            compiler: 'ts:ts-node/register' //interpret step definitions as TypeScript
        },
    };

If we now run npm test it should run our two feature files in parallel against two instances of the ‘Chrome’ browser 😀

Running against multiple browsers in parallel

  1. With the project open in Visual Studio Code, open up the ‘protractor.conf.js’ file again and change the capabilities block to ‘multiCapabilities’ and have one for each browser (Chrome and Firefox), like so…
    multiCapabilities: [ {
        browserName: 'chrome', // set this test browser to 'chrome'
        shardTestFiles: true, // specs are sharded by file when set to true
        maxInstances: 2 // maximum number of browser instances that can run in parallel for this
    }, {
        browserName: 'firefox', // set this test browser to 'firefox'
        shardTestFiles: true, // specs are sharded by file when set to true
        maxInstances: 2 // maximum number of browser instances that can run in parallel for this
    }],

    The whole thing should look similar to below…

    const
        { Photographer, TakePhotosOfFailures } = require('@serenity-js/protractor'),
        { SerenityBDDReporter } = require('@serenity-js/serenity-bdd'),
        { ConsoleReporter } = require('@serenity-js/console-reporter'),
        { ArtifactArchiver } = require('@serenity-js/core');
    
    exports.config = { // export the config so it is accessible by other files
        baseUrl: 'https://www.zagat.com', // // set the base url of our system under test
        directConnect: true, // your test script communicates directly Chrome Driver or Firefox Driver, bypassing any Selenium Server.
        allScriptsTimeout: 120000, // set global protractor timeout to 120 seconds (2 minutes)
        multiCapabilities: [ {
            browserName: 'chrome', // set this test browser to 'chrome'
            shardTestFiles: true, // specs are sharded by file when set to true
            maxInstances: 2 // maximum number of browser instances that can run in parallel for this
        }, {
            browserName: 'firefox', // set this test browser to 'firefox'
            shardTestFiles: true, // specs are sharded by file when set to true
            maxInstances: 2 // maximum number of browser instances that can run in parallel for this
        }],
        specs: ['./features/*.feature'], // set the location of our tests to any and all feature files in the 'features' folder
        framework: 'custom',
        frameworkPath: require.resolve('@serenity-js/protractor/adapter'), // this serenity-js module needs to be called as a custom framework
        onPrepare: () => {
            require('ts-node').register({ // register 'ts-node' to resolve all our package imports
                project: './tsconfig.json' // and handle TypeScript execution smoothly
            });
        },
        serenity: {
            //actors: new Actors(), // instantiate our Actors object so they can use their abilities
            runner: 'cucumber', // tell serenity-js that we want to use the Cucumber runner
            crew: [
                Photographer.whoWill(TakePhotosOfFailures), // or Photographer.whoWill(TakePhotosOfInteractions),
                new SerenityBDDReporter(),
                ConsoleReporter.forDarkTerminals(), // display execution summary in console
                ArtifactArchiver.storingArtifactsAt('./target/site/serenity') // store serenity reports at given location
            ]
        },
        cucumberOpts: {
            require: ['./features/**/*.ts', //require our step definition files
                      './support/**/*.ts'], // also require any necessary files in support folders and sub-folders (e.g. setup.ts)        
                      format: ['snippets:pending-steps.txt'], // workaround to get pending step definitions
            compiler: 'ts:ts-node/register' //interpret step definitions as TypeScript
        },
    };

    If we now run npm test, all our feature files should run in parallel against both Chrome and Firefox simultaneously  😀

Selenium Grid & Docker

We will now move our parallel test run to a Selenium Grid server instead. We will also use Docker to containerise our Selenium Grid (which will consist of a central hub and several chrome and firefox nodes)

Let’s begin 🙂

Docker

Setting up Docker

Docker is a set of platform-as-a-service products that use OS-level virtualisation to deliver software in packages called containers. Containers are isolated from one another and bundle their own software, libraries and configuration files; they can communicate with each other through well-defined channels.

  1. Go to https://www.docker.com/products/docker-desktop and click the button to download Docker Desktop
  2. If this is your first time using Docker, you will be asked to login, click “Signup” to create a new account and follow the simple instructions on their site
  3. Once you’ve created your account, login and download Docker Desktop for your appropriate Operating System (e.g. Mac and Windows)
  4. Follow the normal process to install Docker Desktop on your Operating System
  5. Run Docker Desktop for the first time and work your way through the first time setup flow, which will be similar to below…
    • Click ‘Next’ on the “Welcome to Docker Desktop!” dialog
    • Click ‘OK’ to give privileged access to Docker Desktop (entering your password if necessary)
  6. Now wait for Docker to finish starting up and give confirmation that it is now up and running
  7. Once Docker is up and running, you should see the icon (similar to a whale) in your taskbar, right-click the icon and login to your Docker account

Dockerising our framework

We will now create a docker file and build our framework in a docker container, before pushing it to a Docker hub repository 🙂

Dockerfile

Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image. Using docker build, users can create an automated build that executes several command-line instructions in succession.

To make it easier when creating the Dockerfile, you can use pre-existing images shared with the community on Docker Hub.

Docker Hub

Docker Hub is a service provided by Docker for finding and sharing container images with your team. It provides the following major features: Repositories: Push and pull container images. … Builds: Automatically build container images from GitHub and Bitbucket and push them to Docker Hub.

Creating the Dockerfile
  1. With the project open in Visual Studio Code, choose to create a new file in the project root
    • Name the file ‘Dockerfile’ and click OK
  2. Add the following into the Dockerfile (comments have been included above each line to help explain what we are doing)…
    #Initialise a new build stage and sets the Base Image for subsequent instructions.
    #As such, a valid Dockerfile must start with a FROM instruction. The image can be any valid image.
    #Here we are getting an image running Linux Alpine OS which has NodeJS 13.7 on it
    FROM node:13.7-alpine3.10
    
    #Add a bash shell to the base image defined above
    RUN apk add --no-cache bash
    
    #create a directory called 'app'
    RUN mkdir /app
    
    #The WORKDIR instruction sets the working directory for any RUN, CMD, ENTRYPOINT, COPY and ADD instructions that follow it in the Dockerfile.
    #Here we are setting the app directory we created above as the working directory
    WORKDIR /app
    
    #Copy all files and folders from our project root in to our 'app' working directory
    COPY . /app
    
    #An ENTRYPOINT allows you to configure a container that will run as an executable.
    ENTRYPOINT ["/bin/sh", "-c", "npm test"]

Docker-Compose

Docker-Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure all of your application’s services. Then, with a single command (docker-compose up), you create and start all the services from your configuration. It is just as easy to tear it all down too (docker-compose down).

Creating the Docker-Compose file
  1. With the project open in Visual Studio Code, add a new file to the project root called ‘docker-compose.yaml’
  2. Add the following in the docker-compose.yaml file (comments have been included to help explain what is going on)…
    version: "3.7" #version of docker-compose reference to use
    services: #begin listing our services we want to use for our Grid
      hub:
        image: selenium/hub #get the selenium grid image
        ports:
          - "4444:4444" #set the ports of the hub to 4444:4444
      chrome:
        image: selenium/node-chrome #get the chrome node image
        links:
          - hub #chrome node image requires the hub to work
        environment:
          - HUB_PORT_4444_TCP_ADDR=hub #set the hub port for the chrome node
          - HUB_PORT_4444_TCP_PORT=4444
      firefox:
        image: selenium/node-firefox #get the firefox node image
        links:
          - hub #firefox node image requires the hub to work
        environment:
          - HUB_PORT_4444_TCP_ADDR=hub #set the hub port for the firefox node
          - HUB_PORT_4444_TCP_PORT=4444
      chrome2:
        image: selenium/node-chrome #get the chrome node image
        links:
          - hub #chrome node image requires the hub to work
        environment:
          - HUB_PORT_4444_TCP_ADDR=hub #set the hub port for the chrome node
          - HUB_PORT_4444_TCP_PORT=4444
      firefox2:
        image: selenium/node-firefox #get the firefox node image
        links:
          - hub #firefox node image requires the hub to work
        environment:
          - HUB_PORT_4444_TCP_ADDR=hub #set the hub port for the firefox node
          - HUB_PORT_4444_TCP_PORT=4444
      tests:
        build: . #build from our local Dockerfile
        container_name: tests
        depends_on: # this container only works if the grid is fully up and running
          - chrome
          - firefox
          - chrome2
          - firefox2
        environment:
          - HUB_PORT_4444_TCP_ADDR=hub #set the hub port for the firefox node
          - HUB_PORT_4444_TCP_PORT=4444

If we save the project and then run the command docker-compose up -hub -d from the project root in Terminal/Command Prompt, it will bring up our Selenium Grid as a background process.

We can view our Selenium Grid at http://localhost:4444/grid/console and it will look like below…

Setting the Selenium Server Address in Protractor Config

  1. Open up the ‘protractor.conf.js’ file and add the following so that it uses our localhost Selenium Grid address and also reset the ‘directConnect’ to ‘false’ so it doesn’t bypass our Selenium Server, with the whole file looking like below now…
    const
        { Photographer, TakePhotosOfFailures } = require('@serenity-js/protractor'),
        { SerenityBDDReporter } = require('@serenity-js/serenity-bdd'),
        { ConsoleReporter } = require('@serenity-js/console-reporter'),
        { ArtifactArchiver } = require('@serenity-js/core');
    
    exports.config = { // export the config so it is accessible by other files
        seleniumAddress: 'http://hub:4444/wd/hub', // use our selenium server
        baseUrl: 'https://www.zagat.com', // // set the base url of our system under test
        directConnect: false, // your test script communicates with the Selenium Server now
        allScriptsTimeout: 120000, // set global protractor timeout to 120 seconds (2 minutes)
        multiCapabilities: [ {
            browserName: 'chrome', // set this test browser to 'chrome'
            shardTestFiles: true, // specs are sharded by file when set to true
            maxInstances: 2 // maximum number of browser instances that can run in parallel for this
        }, {
            browserName: 'firefox', // set this test browser to 'firefox'
            shardTestFiles: true, // specs are sharded by file when set to true
            maxInstances: 2 // maximum number of browser instances that can run in parallel for this
        }],
        specs: ['./features/*.feature'], // set the location of our tests to any and all feature files in the 'features' folder
        framework: 'custom',
        frameworkPath: require.resolve('@serenity-js/protractor/adapter'), // this serenity-js module needs to be called as a custom framework
        onPrepare: () => {
            require('ts-node').register({ // register 'ts-node' to resolve all our package imports
                project: './tsconfig.json' // and handle TypeScript execution smoothly
            });
        },
        serenity: {
            //actors: new Actors(), // instantiate our Actors object so they can use their abilities
            runner: 'cucumber', // tell serenity-js that we want to use the Cucumber runner
            crew: [
                Photographer.whoWill(TakePhotosOfFailures), // or Photographer.whoWill(TakePhotosOfInteractions),
                new SerenityBDDReporter(),
                ConsoleReporter.forDarkTerminals(), // display execution summary in console
                ArtifactArchiver.storingArtifactsAt('./target/site/serenity') // store serenity reports at given location
            ]
        },
        cucumberOpts: {
            require: ['./features/**/*.ts', //require our step definition files
                      './support/**/*.ts'], // also require any necessary files in support folders and sub-folders (e.g. setup.ts)        
                      format: ['snippets:pending-steps.txt'], // workaround to get pending step definitions
            compiler: 'ts:ts-node/register' //interpret step definitions as TypeScript
        },
    };

If we now run docker-compose up, it will run all of our tests, in parallel, in docker! Very nice 😀