Running our Tests in Jenkins

Introduction

In the last section, we learnt how to scale up our parallel execution of tests via a dockerised Selenium Grid. In this part, we will now essentially move this in to Jenkins, a build server used for Continuous Integration purposes.

With Jenkins, we can do a lot of cool things, like schedule our tests to run automatically at a set time every day/night, or run every time code is pushed to the master branch of our repository (more on this later).

Let’s get started 😀

Pre-Requisites

I will assume you have followed along with the whole blog series and have Docker setup as well as the dockerised Selenium Grid in your project, as we will be creating a Jenkins server within a Docker container to run on our local machine for this tutorial as well.

I will also assume you have either Homebrew installed on Mac or Chocolatey on Windows.

Setup

Download Jenkins (docker image)

macOS / Linux

  1. Open up a Terminal window and go to your home directory… 
    cd
  2. Next, paste in the following command to pull the Jenkins blue ocean image and run it… 
    docker run \
 -d \
 -p 8080:8080 \
 -p 50000:50000 \
 -v "$PWD/jenkins:/var/jenkins_home" \
 jenkinsci/blueocean
  • -d flag runs the above command in the background, so we can still use the Terminal window/tab whilst Jenkins is running
  • -p flag is for port mapping (e.g. maps the port 8080 of the container to port 8080 of the local host)
  • -v flag is for volume mapping (e.g. maps a local Jenkins directory in the current working directory to the Jenkins home directory in the container)

Windows

  1. Open up a CMD prompt and go to the home directory… 
    cd
  2. Next, paste in the following command to pull down the Jenkins blue ocean image and run it… 
    docker run ^
 -d ^
 -p 8080:8080 ^
 -p 50000:50000 ^
 -v "$PWD/jenkins:/var/jenkins_home" ^
 jenkinsci/blueocean
  • -d flag runs the above command in the background, so we can still use the Terminal window/tab whilst Jenkins is running
  • -p flag is for port mapping (e.g. maps the port 8080 of the container to port 8080 of the local host)
  • -v flag is for volume mapping (e.g. maps a local Jenkins directory in the current working directory to the Jenkins home directory in the container)

Unlocking Jenkins

When you first access a new Jenkins instance, you will be asked to unlock it using an automatically-generated password. Our downloaded and dockerised Jenkins instance can be found at http://localhost:8080 . In order to retrieve the password to unlock it, we can do the following…

  1. In the CMD or Terminal window, enter the following command to list our docker containers…
    docker psYou should see output similar to below…

    CONTAINER ID        IMAGE                 COMMAND                  CREATED             STATUS              PORTS                                              NAMES
15df6769a1ab        jenkinsci/blueocean   "/sbin/tini -- /usr/…"   10 minutes ago      Up 10 minutes       0.0.0.0:8080->8080/tcp, 0.0.0.0:50000->50000/tcp   zen_turing
  2. Using the container name output above, enter the following command to get the initial password to unlock Jenkins (this can also be found in the $PWD/jenkins/secrets/ folder on your local machine due to the volume mapping we did)… 
    docker exec <container_name> cat /var/jenkins_home/secrets/initialAdminPassword

    The password should then be output in the CMD or Terminal window.

  3. Copy and paste the password in to your local Jenkins instance and click ‘Continue’ to unlock it

Setting up Jenkins

  1. Next, select to ‘Install suggested plugins’ and wait for everything to finish
  2. Enter in details that you want to use for your username, password, full name and email address etc. and then click ‘Save and Continue’
  3. Leave the default value in the ‘Jenkins URL’ field and click ‘Save and Finish’
  4. Click ‘Start using Jenkins’ to finish setup 😀

Creating a new Node in Jenkins

We will now create a new node / agent to execute our builds on, instead of relying on Master which currently does not contain the things we need like Docker, Docker-Compose etc.

  1. On the Jenkins landing page when logged in, click “Build Executor Status”
  2. Click “New Node”
    1. Name the node something appropriate (e.g. ‘Docker_1’)
    2. Select the ‘Permanent Agent’ radio button and click ‘OK’
  3. For ‘# of Executors’, input a suitable number (e.g. 3)
  4. For ‘Remote root directory’, input the absolute directory path to the Jenkins root that you volume mounted to when running the Jenkins container (e.g. ‘/Users/username/jenkins’)
  5. For ‘Launch Method’, because we are running Jenkins locally without an available open SSH server, let’s select ‘Launch agent by connecting it to the master’
  6. Click ‘Save’ and you should then see the newly created node displayed in the nodes table
  7. Click on the name of the node you just created
  8. Click the ‘agent.jar’ hyperlink
  9. Open up a new Terminal window/tab and change directory (cd) in to the folder where the ‘agent.jar’ file got downloaded to, like below… 
    cd Downloads
  10. run the command shown on the Jenkins page to run the agent.jar file, similar to below… 
    java -jar agent.jar -jnlpUrl http://localhost:8080/computer/Docker_1/slave-agent.jnlp -secret f94e1c3f062d21d6394f1f560ce019ac9d9ce29049bd832b011e59521fd57f95 -workDir "/Users/username/jenkins"

The newly created node / agent should now be active and running successfully 😀

Git Repository

GitLab Account and Project Creation

We will now create a Git repository on Gitlab.com to store our project. Later, we can then connect our Git repository to Jenkins and watch it do its magic!

  1. Go to https://gitlab.com/users/sign_in#register-pane and create a valid account and then login if necessary
  2. Once logged in, click ‘New Project’
    1. Give your project a suitable name (e.g. ‘java-selenium-grid-docker’)
    2. Select a suitable ‘Project URL’ and ‘Project Slug’
    3. Choose your visibility level (e.g. Public)
    4. Choose whether you want to initialize your git repository with a readme.md file or not
  3. Click ‘Create Project’

Install Git locally

macOS

Open up a Terminal window and enter the following command…

  1. brew install git

Windows

  1. Open up an elevated CMD prompt window (right-click CMD prompt and ‘Run as Administrator’) and enter the following command… 
    choco install git

    If prompted, enter in ‘A’ to select All for the scripts

Push your local repository to Git

  1. Open up a Terminal or CMD prompt window and change directory (cd) in to the root of your project… 
    cd path/To/Project/Root
  2. Initialize an empty Git repository with the following command… 
    git init
  3. Add the remote origin of your git repository (shown on your project’s empty Gitlab page if you left it open), similar to below… 
    git remote add origin https://gitlab.com/testifyqa/java-selenium-docker.git
  4. Add everything in the project to the Git staging area… 
    git add .
  5. Add an initial commit message and commit your staged changes… 
    git commit -m "Initial commit"
  6. Push your committed changes to your Git repository… 
    git push -u origin master

Now we have our project in Gitlab, we can continue on with Jenkins 😀

Jenkins Pipelines

Introduction

Pipelines are Jenkins jobs enabled by the Pipeline (formerly called “workflow”) plugin and built with simple text scripts that use a Pipeline DSL (domain-specific language) based on the Groovy programming language.

Pipelines leverage the power of multiple steps to execute both simple and complex tasks according to parameters that you establish. Once created, pipelines can build code and orchestrate the work required to drive applications from commit to delivery.

Our pipeline is defined in a ‘Jenkinsfile’ in the root of our project. Jenkins picks up the instructions in the Jenkinsfile based on a schedule we set (e.g. at a set time every day/night, or when a merge or push is made to master).

Creating the Jenkinsfile

  1. With the project open in IntelliJ, right-click on the project name/root in the Project Explorer pane and select ‘New’ -> ‘File’
    1. Name the file ‘Jenkinsfile’ and press <Return> key
    2. If prompted to add the file to Git, click ‘Add’
  2. Add the following in the Jenkinsfile (I have added comments to try and explain what’s going on, for more info though, check out https://jenkins.io/doc/book/pipeline/jenkinsfile/)… 
    pipeline {
    agent any // use any available agent in Jenkins

    stages {
        stage('Clone repo in to Jenkins instance') {
            steps {
                deleteDir()
                checkout scm // checkout and clone our git repo in to the Jenkins instance
            }
        }
        stage('Pull latest Docker image') {
            steps {
                // replace 'chronicideas/selenium-testifyqa-docker' with your image name (you can grab it from your docker-compose file)
                sh 'docker pull chronicideas/selenium-testifyqa-docker' // pull down your latest docker image you made in previous parts
            }
        }
        stage('Start Grid') {
            steps {
                sh 'docker-compose up -d hub chrome firefox' // bring the grid up in the background so that Terminal window can still be used
            }
        }
        stage('Run Tests') {
            steps {
                sh 'docker-compose up base-tests search-tests' // run all tests
            }
        }
        stage('Tear down') {
            steps {
                sh 'docker-compose down' // bring the docker containers down
            }
        }
    }
}

Creating the Jenkins Job

We will now create a job in Jenkins that will read the Jenkinsfile in our project when we click ‘Build Now’ or when a push is made to master.

We can actually do a lot of things with Jenkins, including have it build and push our project to Docker Hub before separately pulling that image and running it when executing our tests.  However, for this guide, we will assume that the docker image is built and pushed manually and then just use the Jenkinsfile to pull the latest image and run the tests 😀

  1. On the Jenkins landing page once logged in, click ‘create new jobs’
  2. Enter a suitable item name (e.g. “docker-selenium-grid-tests”)
  3. Select ‘Pipeline’ and click ‘OK’
  4. Scroll down to the ‘Pipeline’ section and select ‘Pipeline script from SCM’ in the ‘Definition’ dropdown
  5. In the ‘SCM’ dropdown select ‘Git’
  6. In the ‘Repository URL’ field, enter the URL of your GitLab repository you created earlier (add ‘.git’ to end of URL if you get any issues)
  7. Click ‘Save’

If we now click ‘Build Now’, Jenkins will trigger a build and run our tests in parallel in a dockerised Selenium Grid!! 😀

Note: If you add plugins to Jenkins you can add more functionality like getting Jenkins to trigger builds automatically when you make a push to master in your GitLab repository etc.

Liked it? Take a second to support Thomas on Patreon!

Previous Article

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.