Parallel Test Execution (Maven)

Introduction

Right now, when we run all of our tests in our framework, they are run sequentially (e.g. one at a time).  This is arguably ok for now as we only have a total of 5 test scenarios (4 base scenarios and 1 proper test), running against only one Android device (emulator) that we setup.

But let’s imagine we have loads of tests in our framework, and those tests need testing across multiple mobile devices. If we ran all those tests against one mobile device at a time, the time it would take to complete would be relatively long.

This is where parallel test execution comes in. By using multiple forks or threads, we can run our tests against multiple mobile devices concurrently (at the same time) to reduce the total amount of time it takes to execute all the tests in the framework.

Back in the earlier days of Appium, we would need to run a separate Appium server for each mobile device we want to run in parallel. Luckily, this is no longer the case, as we can now run only one Appium server that handles multiple sessions for all of our mobile devices.

In today’s modern age, most computers have multiple processors, which we can use to our advantage to run tests on a few different forks or threads at the same time.  If we want to run on loads of forks or threads without making the computer unusable, we can scale up using something like Selenium Grid, and then scale up even further with a cloud based service like SauceLabs. We will cover both of these in later parts of this blog series.

Let’s get started 😀

Running our tests against multiple mobile devices concurrently

Due to the fact that we cannot run multiple tests against the same mobile device at the same time, our best options for parallel test execution approach are either:

  • Distribution – Feature files are distributed almost equally on all devices available during a run
  • Fragmentation – All Features are run across all available devices

Due to the currently very small size of our test framework, we will go with the Fragmentation approach.

Currently the way our framework is setup, is that in our pom.xml file (Maven), we specify the testng.xml file that contains all of our test classes.

We will be wanting to run our tests in parallel on the Cucumber level, utilising the Maven-SureFire plugin, rather than the TestNG level, due to the simple fact that it’s a lot quicker and easier to get it setup. This means we no longer need the testng.xml file!

  1. So, firstly, with the project open in IntelliJ, right-click on the ‘testng.xml’ file and select ‘Delete’ and then ‘OK’

Adding another Android device (emulator)

Creating a new Android emulator

  1. Launch Android Studio
    • Select ‘Start a new Android Studio Project’
    • Leave all the options as they are by default in the ‘Create New Project’ wizard and keep clicking ‘Next’
    • Click ‘Finish’ on the last screen of the wizard
  2. In Android Studio, advance to ‘Tools’ -> ‘AVD Manager’
  3. Click ‘Create Virtual Device’
    • Select a suitable device definition (e.g. Phone – Nexus 6P) and click ‘Next’
    • Select a suitable system image / Android OS (e.g. Marshmallow – API Level 27) and click ‘Next’ (usually latest is the best to choose, make sure it is compatible with Appium and download the image if necessary)
    • Give the Android Virtual Device (AVD) a suitable name (e.g. Nexus6pMarshmallow) and click ‘Finish’
      • You can now click the ‘Play’ icon and open the virtual device if you want and play around with it (e.g. try opening Browser in it or another app)
  4. From now on, you can also open up the AVD via Terminal (or command prompt).  Open up Terminal and type in the following commands…
    cd $ANDROID_HOME/platform-tools
    emulator -avd yourAvdName (replace ‘yourAvdName’ with name of the AVD you made)

Adding another ‘AndroidDriver’ for the new mobile device

  1. With the project open in IntelliJ, open up the ‘AndroidAppDriver’ class and add a new ‘public static AndroidDriver’ method to load the new mobile device emulator you just created, like below…

    The whole thing should look similar to below…
Refactoring the ‘AndroidAppDriver’ class

You might notice here that our two ‘AndroidDriver’ methods have a lot of duplicated code, making it not very DRY (Do not Repeat Yourself). We should fix this by instead creating a shared method where we can pass in arguments for our DesiredCapabilities.

To safely run multiple sessions in the same Appium Server, each Android session needs to be run on a different ‘System Port’:

  • systemPort: to communicate to the UiAutomator2 process, Appium utilises an HTTP connection which opens up a port on the host system as well as on the device. The port on the host system must be reserved for a single session, which means that if you’re running multiple sessions on the same host, you’ll need to specify different ports here (for example 8200 for one test thread and 8201 for another). Ports 8200-8299 are used for UiAutomator2, where as ports 8300-8399 are given for Android Espresso.

We also HAVE TO to declare the udid for each device (even emulators). This can be found by starting up an Android emulator and entering into Terminal (or command prompt)…  adb devices . If we don’t include this capability, the driver will attempt to use the first device in the list returned by ADB. This could result in multiple sessions targeting the same device, which is not good.

  1. Let’s make our generic ‘loadDriver’ method that will take in arguments that we pass in to it (‘AVD’ and ‘DeviceName’ are the same so we only need to pass one String for those, and then another String for ‘udid’ and one more int for ‘systemport’)…

    We can now get rid of our other two methods. The whole thing should then look like below…

Refactoring our AppiumServer class

Because we can now run multiple sessions in a single Appium server, we only need to startup the Appium server if it is not already running. Let’s refactor our AppiumServer class to check for this and only start it up if it is not already running 🙂

  1. Open up the ‘AppiumServer’ class in the ‘utils.appium’ package and refactor it to match below…

Refactoring the DriverController class

Because we now have only one generic method that we pass arguments into for loading the Android driver sessions, we only need one generic method in the DriverController class to start the driver session  by calling the loadDriver() method with the arguments we pass into it.

  1. Open up the ‘DriverController’ class in the ‘utils.appium’ package and refactor it to look like below…

Forking

In order for our tests to successfully run across multiple mobile devices in parallel, we need to run multiple forks, with each fork running our tests against a particular device in one session.

It is also possible to run each session in a thread (e.g. by putting our Drivers or DriverController instance in a ThreadLocal<>), however running them in forks is easier.  Each fork uses its own JVM in contrast to Threads. Running multiple JVM’s uses up more resources of course, but is a lot quicker and easier to get started with.

TestRunners

To run separate sessions in each fork, we first need to create TestRunner classes to run all of our feature files against different specified mobile devices. Let’s do that now.

  1. In IntelliJ, right-click on the ‘TestRunner’ class and select ‘Refactor’ -> ‘Rename’
    • Rename the class to ‘Nexus5xTestRunner’, and click ‘Refactor’ (and then ‘Do Refactor’ if necessary)
  2. Open up the ‘Nexus5xTestRunner’ class and edit it to match below…

    Now let’s make another similar TestRunner for our ‘Nexus6P’ mobile device (emulator)
  3. In IntelliJ, right-click on the ‘Nexus5xTestRunner’ class in the project explorer and select ‘Copy’
  4. Click on the ‘Nexus5xTestRunnerClass’ in the project explorer again and press Ctrl/CMD+V to paste
  5. Rename the copied class to ‘Nexus6pTestRunner’ and click OK
  6. Open up the ‘Nexus6pTestRunner’ class and refactor it to match below…

Editing the pom.xml

We will now edit the Maven-SureFire plugin in our pom.xml to run each of our TestRunner’s in parallel, in different forks.

Editing the ‘maven-surefire-plugin’
  1. Open up the pom.xml file and edit the ‘maven-surefire-plugin’ so it matches below…

Again, notice the following that we have included…

  • <forkCount> – This specifies the number of forks we run concurrently.  We have set this to 2 as we currently only have 2 TestRunners for each device session
  •  <reuseForks> – This specifies if we want to reuse forks after one has finished running a TestRunner class
  • <include> – This specifies the location and files to include in our forks.  We have used a glob pattern to specify to include all class files that end with ‘TestRunner’ within any child directory of any parent directory (relative from the project root)

Cleaning up the CucumberHooks

Because we now startup and teardown the appropriate drivers in our TestRunner classes, we only really need the CucumberHooks when running a test locally or debugging. Therefore, we should change our tags in there to reflect this.

  1. Open up the CucumberHooks class and refactor so it’s similar to below…

Running our Tests

  1. For simplicity, remove all tags in your Feature files and place both ‘@Nexus5xOreo’ and ‘@Nexus6pMarshmallow’ tags at the top of the ‘NavigationScenarios.feature’ file, like below…
  2. Open up Terminal (or Command line) and change directory (cd) into your project root
  3. Enter  mvn test  to run all the tests in parallel 😀
Liked it? Take a second to support Thomas on Patreon!

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.