Quick Start - Azure Pipelines

Quick Start - how to use Diffblue Cover to write tests in Azure Pipelines CI

Overview

This guide explains how to use Diffblue Cover to write tests in Azure Pipelines CI. This guide assumes that you have:

• A Maven or Gradle project that:

  • Compiles

  • Does not have non-compiling or failing tests

  • Is stored in a Git repository which you are able to create a new Azure Pipeline YAML for

• A basic understanding of Azure Pipelines

• The ability to add secret variables to a pipeline

• Diffblue Cover stored in the cloud for download along with the license key. See Installation.

To integrate Diffblue Cover into your CI pipeline, we will guide you through creating a pipeline YAML that:

1. Builds your project

2. Downloads and activates Diffblue Cover CLI

3. Runs Diffblue Cover to create tests

4. Commits the created tests to a branch

The following sections provide more details for each of the above steps.

1. Building the project

Add a new pipeline to your project's CI. In the example below the pipeline is triggered on any pull request against the main branch. Make sure to check, and if necessary modify, the command to build your project. Running the project’s tests is not required, and you will save time by skipping them, but they do need to compile and pass.

name: Cover

trigger: none
pr:
  branches:
    include:
    - main

pool:
  vmImage: 'ubuntu-latest'

variables:
  BASH_ENV: "~/.profile"

jobs:
- job: update_diffblue_tests
  displayName: Update Diffblue Tests

  steps:
  - checkout: self
    persistCredentials: true

  - script: |
      git checkout "$(System.PullRequest.SourceBranch)"
    displayName: 'Checkout Branch'

  - script: |
      mvn --batch-mode --no-transfer-progress clean install -DskipTests
    displayName: 'Build Project'

2. Downloading and activating Diffblue Cover CLI

You need to give the CI run access to the Diffblue Cover files and activate the dcover license in order to write tests.

This guide assumes that you have a URL with the Diffblue Cover CLI release zip and the license key for online activation during the CI run. See Installation. If your license allows it you may wish to install Diffblue Cover with offline activation. See Licensing.

Add a new variable with the name DIFFBLUE_COVER_URL, set the value to the URL of the Diffblue Cover CLI release zip file and tick the "Keep this value secret" box. Then add a second variable with the name DIFFBLUE_COVER_LICENSE_KEY. Set the value to your Diffblue Cover license key and make this a secret variable as well.

Append the code for getting, unzipping and activating dcover to your pipeline YAML.

  - script: |
      mkdir dcover
      cd dcover
      curl --silent --show-error --location --output "diffblue-cover-cli.zip" "$(DIFFBLUE_COVER_URL)" 
      unzip -q "diffblue-cover-cli.zip"
      rm --force "diffblue-cover-cli.zip"
      echo "export PATH=$PATH:$PWD/" >> ~/.profile
      source ~/.profile
    displayName: 'Download Diffblue Cover'
  
  - script: |
      dcover activate "$(DIFFBLUE_COVER_LICENSE_KEY)"
    displayName: 'Activate Diffblue Cover'

This will put the Diffblue Cover files into the dcover directory in the root of the workspace. The Diffblue Cover files contain a script to invoke dcover which has the relative path dcover/dcover. Add this to your path so that you can call Diffblue Cover as dcover. In order to add Cover to your path ensure you include the "BASH_ENV" variable from the first step.

Push the changes so this workflow runs - ensure that you can see the successful activation of dcover in your "Activate Diffblue Cover" step before moving on. You will see a line starting with "Successfully activated key" if this was successful. If your Diffblue Cover did not successfully activate, please see Licensing or contact Diffblue Support.

3. Running Diffblue Cover CLI to create tests

Now that Diffblue Cover is running in Azure Pipelines, you can use it to write tests. Append the following to your pipeline YAML. Note that the --batch option makes the output more suitable for CI, as it ensures the CI logs are not cluttered with progress bar output.

  - script: dcover create --batch
    displayName: 'Create tests using Diffblue Cover'

Push the changes so this workflow runs. Once successfully complete, you should expect to see output that looks like this in your "Create tests using Diffblue Cover" step:

INFO  Found 7 callable methods in 2 classes
INFO
INFO  Creating tests:
INFO  ---------------
...
INFO  All 5 created tests were successfully validated.

If you don't see this output, the call may need small modification for your project or dependencies adding until it works. The output gives you warnings along the way to guide you. See CLI Commands for more information.

Depending on the size of your module/project, creating tests could take a while. You may wish to restrict test creation to a single class by specifying its fully qualified name:

dcover create com.somepackage.SomeClass --batch

4. Committing the created tests to a branch

To see these new tests in the project you'll need to commit them and push back to the repository. You'll need to configure Git credentials to commit. We recommend creating a service account for this.

  - script: |
      git config user.name db-ci-bot
      git config user.email [email protected]
    displayName: 'Config git user'

To commit the tests append the following to your pipeline YAML. This will check for any changes to Diffblue tests, add them to a commit and push to your branch.

  - script: |
      if [ -n "$(git status --short **/*DiffblueTest.java)" ]; then 
        git add **/*DiffblueTest.java
        git commit --message "Update Unit Tests for $(git rev-parse --short HEAD)"
        git push --set-upstream origin
      else
        echo "Nothing to commit"
      fi
    displayName: Stage, commit and push unit tests

Please note - be careful not to create an infinite CI loop here. This is because pushing a commit with updated tests will re-trigger the pipeline. To avoid this becoming a loop we recommend checking the author of each commit to ensure you are not creating tests for a commit authored by your Diffblue service account. This can be done by appending the following to the start of your pipeline YAML.

  - script: |
      LAST_NON_BOT_COMMIT="$(git rev-list -1 --author='^(?!db-ci-bot).*$' --perl-regexp HEAD --no-merges)"
      echo "Last non bot commit is $LAST_NON_BOT_COMMIT"
      LAST_COMMIT="$(git rev-list HEAD -1 --no-merges)"
      echo "Last commit is $LAST_COMMIT"
      if [[ "$LAST_NON_BOT_COMMIT" == "$LAST_COMMIT" ]]
      then
        echo "##vso[task.setvariable variable=last_bot_commit]0"
      else
        echo "##vso[task.setvariable variable=last_bot_commit]1"
      fi
    displayName: 'Check if last commit was bot commit'

Then append this condition to all of the subsequent jobs:

  condition: eq(variables.last_bot_commit, 0)

Your final pipeline YAML should look similar to:

name: Cover

trigger: none
pr:
  branches:
    include:
    - main

pool:
  vmImage: 'ubuntu-latest'

variables:
  BASH_ENV: "~/.profile"

jobs:
- job: update_diffblue_tests
  displayName: Update Diffblue Tests

  steps:
  - checkout: self
    persistCredentials: true

  - script: |
      git checkout "$(System.PullRequest.SourceBranch)"
    displayName: 'Checkout Branch'

  - script: |
      LAST_NON_BOT_COMMIT="$(git rev-list -1 --author='^(?!db-ci-bot).*$' --perl-regexp HEAD --no-merges)"
      echo "Last non bot commit is $LAST_NON_BOT_COMMIT"
      LAST_COMMIT="$(git rev-list HEAD -1 --no-merges)"
      echo "Last commit is $LAST_COMMIT"
      if [[ "$LAST_NON_BOT_COMMIT" == "$LAST_COMMIT" ]]
      then
        echo "##vso[task.setvariable variable=last_bot_commit]0"
      else
        echo "##vso[task.setvariable variable=last_bot_commit]1"
      fi
    displayName: 'Check if last commit was bot commit'

  - script: |
      mvn --batch-mode --no-transfer-progress clean install -DskipTests
    displayName: 'Build Project'
    condition: eq(variables.last_bot_commit, 0)

  - script: |
      mkdir dcover
      cd dcover
      curl --silent --show-error --location --output "diffblue-cover-cli.zip" "$(DIFFBLUE_COVER_URL)" 
      unzip -q "diffblue-cover-cli.zip"
      rm -f "diffblue-cover-cli.zip"
      echo "export PATH=$PATH:$PWD/" >> ~/.profile
      source ~/.profile
    displayName: 'Download Diffblue Cover'
    condition: eq(variables.last_bot_commit, 0)
  
  - script: |
      dcover activate "$(DIFFBLUE_COVER_LICENSE_KEY)"
    displayName: 'Activate Diffblue Cover'
    condition: eq(variables.last_bot_commit, 0)

  - script: |
      dcover create --batch
    displayName: 'Creating unit tests using Diffblue Cover'
    condition: eq(variables.last_bot_commit, 0)

  - script: |
      git config user.name db-ci-bot
      git config user.email [email protected]
    displayName: 'Config git user'
    condition: eq(variables.last_bot_commit, 0)

  - script: |
      if [ -n "$(git status --short **/*DiffblueTest.java)" ]; then 
        git add **/*DiffblueTest.java
        git commit --message "Update Unit Tests for $(git rev-parse --short HEAD)"
        git push --set-upstream origin
      else
        echo "Nothing to commit"
      fi
    displayName: Stage, commit and push unit tests
    condition: eq(variables.last_bot_commit, 0)