Cover Pipeline for GitLab integrates the power of Diffblue Cover directly into your GitLab pipeline to autonomously write Java unit tests for your projects on merge requests.

• Automatically write a baseline unit test suite for your projects.

• Automatically write new unit tests for new code.

• Automatically update existing unit tests in your code.

• Automatically remove existing unit tests in your code when they’re no longer required.

Poor test coverage, or the complete lack of unit tests, acts as a blocker to CI implementation. Diffblue's reinforcement learning AI platform removes this blocker, autonomously writing and updating Java unit tests for your projects directly in your GitLab pipeline, helping you to save developer time, increase test coverage, and catch regressions.

On first use, Diffblue Cover will create your full test suite for your entire project. On subsequent code-change merge requests, these tests will be automatically updated to reflect the new behavior, allowing you and your development team to catch regressions and unplanned behavior changes.

eLearning

Check out our introductory video for an overview of Cover Pipeline for GitLab and a brief demonstration of how to install, activate, and use Diffblue Cover directly within your GitLab pipeline. [4min]

Implementing Diffblue Cover into your pipeline is a straightforward process. In brief, we’ll:

1. Find and configure the Diffblue Cover integration.

2. Set up a pipeline for the sample project using the GitLab pipeline editor and the Diffblue Cover pipeline template. Note that the Quick Start - General describes how to configure Diffblue Cover in CI without using the integration or pipeline template.

3. Create the full baseline unit test suite for a sample project.

eLearning

Short on time? If you'd like a fast-track experience check out our getting started video. [5 min]

1. Configure Diffblue Cover

1. Clone the sample project from the https://github.com/diffblue/demo-spring-petclinic Diffblue repo to your GitLab account.

2. From the project in GitLab (either the sample Spring PetClinic project, or your own project) go to Settings > Integrations, find Diffblue Cover, and click Configure.

3. In the configuration screen, update the following (some items may be pre-selected):

• Check/tick the Active box.

• Enter your Diffblue Cover License key provided in your welcome email or by your organization. If needed, click the Try Diffblue Cover link to sign up for a free trial.

• Enter details of your GitLab access token (Name and Secret) to allow Diffblue Cover to access your project. In general, use a GitLab Project Access Token with a Developer role, plus api and write_repository scopes/permissions. If necessary you can use Group Access or Personal Access Tokens, again with api and write permissions. See the GitLab docs links below for full details on how to create an access token.

1. When you're done, click Save changes to apply your updates. Your Diffblue Cover integration is now Active and ready for use in your project.

2. Set up a pipeline

Here we'll create a merge request pipeline for the project that will download the latest version of Diffblue Cover, build the project, write Java unit tests for the project, and commit the changes to the branch.

1. From the project in GitLab go to Build > Pipeline editor and click Configure pipeline to create/edit the .gitlab-ci.yml file.

2. Copy the sample template below and paste this into the editor over-writing the default template created by GitLab.

1. When we commit these changes we'll use the commit to create a new branch and create a new merge request. So before you click Commit, update the following:

• Update the Commit message with an appropriate description such as "Update .gitlab-ci.yml file".

• Update the Branch field with an appropriate name such as "add-diffblue-cover-pipeline".

• Check/tick Start a new merge request.

1. When you're ready, click Commit changes and go to #3.-create-the-baseline-unit-test-suite below.

For reference, an example Diffblue Cover pipeline template is shown below.

# This template is provided and maintained by Diffblue.
# You can copy and paste this template into a new `.gitlab-ci.yml` file.
# This template is designed to be used with the Cover Pipeline for GitLab integration from Diffblue.
# It will download the latest version of Diffblue Cover, build the associated project, and
# automatically write Java unit tests for the project.
# Note that additional config is required:
# https://docs-proxy.diffblue.io/features/cover-pipeline/cover-pipeline-for-gitlab
# You should not add this template to an existing `.gitlab-ci.yml` file by using the `include:` keyword.
#
# To contribute improvements to CI/CD templates, please follow the Development guide at:
# https://docs.gitlab.com/ee/development/cicd/templates.html
# This specific template is located at:
# https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Diffblue-Cover.gitlab-ci.yml

variables:
  # Configure the following via the Diffblue Cover integration config for your project, or by
  # using CI/CD masked Variables.
  # For details, see https://docs-proxy.diffblue.io/features/cover-pipeline/cover-pipeline-for-gitlab

  # Diffblue Cover license key: DIFFBLUE_LICENSE_KEY
  # Refer to your welcome email or you can obtain a free trial key from
  # https://www.diffblue.com/try-cover/gitlab

  # GitLab access token: DIFFBLUE_ACCESS_TOKEN, DIFFBLUE_ACCESS_TOKEN_NAME
  # The access token should have a role of Developer or better and should have
  # api and write_repository permissions.

  # Diffblue Cover requires a minimum of 4GB of memory.
  JVM_ARGS: -Xmx4g

stages:
  - build

diffblue-cover:
  stage: build

  # Select the Cover CLI docker image to use with your CI tool.
  # Tag variations are produced for each supported JDK version.
  # Go to https://hub.docker.com/r/diffblue/cover-cli for details.
  # Note: To use the latest version of Diffblue Cover, use one of the latest-jdk<nn> tags.
  # To use a specific release version, use one of the yyyy.mm.dd-jdk<nn> tags.
  image: diffblue/cover-cli:latest-jdk17

  # Diffblue Cover currently test creation only supports running on merge_request_events.
  rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'

  # Diffblue Cover log files are saved to a .diffblue/ directory in the pipeline artifacts,
  # and are available for download once the pipeline completes.
  artifacts:
    paths:
      - "**/.diffblue/"

  script:

    # Diffblue Cover requires the project to be built before creating any tests.
    # If a simple maven or gradle build is insufficient for your project, please
    # build the project here and remove the build argument from the next step
    # custom built command here

    # Diffblue Cover commands and options to run.
    #   dcover – the core Diffblue Cover command
    #   ci – enable the GitLab CI/CD integration via environment variables
    #   activate - activate the license key
    #   build - build the project
    #   validate - remove non-compiling and failing tests
    #   create - create new tests for your project
    # For detailed information on Cover CLI commands and options, see
    # https://docs-proxy.diffblue.io/features/cover-cli/commands-and-arguments
    - dcover
        ci
        activate
        build
        validate
        create

    # Diffblue Cover will also respond to specific project labels:
    #   Diffblue Cover: Baseline
    #     Used to mark a merge request as requiring a full suite of tests to be written.
    #     This overrides the default behaviour where Cover will only write tests related
    #     to the code changes already in the merge request. This is useful when running Diffblue
    #     Cover for the first time on a project and when new product enhancements are released.
    #   Diffblue Cover: Skip
    #     Used to mark a merge request as requiring no tests to be written.
    

3. Create the baseline unit test suite

1. On the New merge request screen, update Title with an appropriate merge request title such as Add Cover pipeline and create baseline unit test suite.

1. When you're ready, click Create merge request.

2. The merge request pipeline will now run Diffblue Cover to create the baseline unit test suite for the project. Once complete, go to the src/test folders in the project repo to see the unit tests created by Diffblue Cover (suffixed with *DiffblueTest.java).

Subsequent code changes

When performing subsequent code changes to a project, the merge request pipeline will run Diffblue Cover but will only update the associated tests. The resulting diff can then be analyzed to check the new behavior, catch regressions, and spot any unplanned behavioral changes to the code. This workflow is illustrated further in GitLab workflow.

Next steps

This topic demonstrates some of the key features of Cover Pipeline for GitLab and how to use the integration within a pipeline. The wider and deeper functionality, provided through dcover commands in the pipeline template, can be implemented to extend your unit test capabilities even further.

When using the Diffblue Cover pipeline template with your own project and existing pipeline file, add the Diffblue template content to your file and modify as needed - see Configuration for details. Note that only specific dcover commands and arguments are currently supported for use with Cover Pipeline for GitLab, as detailed in Configuration.

Once you're familiar with the topics covered under Cover Pipeline for GitLab, check out the Related topics to expand your knowledge further.

Integrating Diffblue Cover into your CI/CD pipeline supports the software development workflow to provide AI automation of unit tests. The following provides an illustrative development workflow, with and without Cover Pipeline for GitLab, covering key steps to illustrate differences in workflow.

There are four key components for configuring Cover Pipeline for GitLab:

• Labels: Pre-defined Diffblue Cover project labels to determine test requirements on merge requests (full baseline, update, or skip).

• Integration Config: Direct license key and access token config using the Diffblue Cover integration page on GitLab.

• Pipeline Config: Configure the Diffblue Cover section of your pipeline including Cover version, build config, and Cover commands to execute on merge requests.

• Masked Variables: Define the Diffblue Cover license key and GitLab access token using GitLab masked variables. This avoids the need to configure the integration directly from the GitLab Integrations page.

Labels

When writing tests, Diffblue Cover will respond to specific project labels:

• Diffblue Cover: Baseline Used to mark a merge request as requiring a full suite of tests to be written. This overrides the default behavior where Cover will only write tests related to the code changes already in the merge request. The baseline label is applied automatically when running Diffblue Cover for the first time on a project (to create the initial baseline test suite). On subsequent runs, you can select this label within a merge request, if required - useful when you want to refresh your entire test suite, such as when new product enhancements are released (to update the entire test suite with the latest enhancements).

• Diffblue Cover: Skip Used to mark a merge request as requiring no tests to be written. The skip label is useful when performing merge requests that have no impact on unit tests, such as updates to comments only.

These labels are created automatically in GitLab when running Diffblue Cover for the first time on a project.

Integration config

You can configure the Diffblue license key and associated GitLab access token directly, using the Diffblue Cover integration page on GitLab. Note that, if preferred, you can use GitLab masked variables to define these properties - see Masked variables.

1. From your project in GitLab go to Settings > Integrations, find Diffblue Cover, and click Configure.

2. In the configuration screen, update the following (some items may be pre-selected):

• Check/tick the Active box.

• Enter your Diffblue Cover License key provided in your welcome email or by your organization. If needed, click the Try Diffblue Cover link to sign up for a free trial.

• Enter details of your GitLab access token (Name and Secret) to allow Diffblue Cover to access your project. In general, use a GitLab Project Access Token with a Developer role, plus api and write_repository scopes/permissions. If necessary you can use Group Access or Personal Access Tokens, again with api and write permissions. See the GitLab docs links below for full details on how to create an access token.

1. When you're done, click Save changes to apply your updates. Your Diffblue Cover integration is now now Active and ready for use in your projects.

Pipeline config

Update or create the .gitlab-ci.yml file for your project to configure the Diffblue Cover section of your CI/CD pipeline.

1. From your project in GitLab go to Build > Pipeline editor and click Configure pipeline to create/edit the .gitlab-ci.yml file.

2. Copy the Diffblue Cover pipeline template below Into your project .gitlab-ci.yml pipeline file, and update if needed. Configurable properties are listed below - for everything else, please leave these set to the values defined in the Diffblue template.

For reference, the Diffblue Cover pipeline template is shown below.

# This template is provided and maintained by Diffblue.
# You can copy and paste this template into a new `.gitlab-ci.yml` file.
# This template is designed to be used with the Cover Pipeline for GitLab integration from Diffblue.
# It will download the latest version of Diffblue Cover, build the associated project, and
# automatically write Java unit tests for the project.
# Note that additional config is required:
# https://docs-proxy.diffblue.io/features/cover-pipeline/cover-pipeline-for-gitlab
# You should not add this template to an existing `.gitlab-ci.yml` file by using the `include:` keyword.
#
# To contribute improvements to CI/CD templates, please follow the Development guide at:
# https://docs.gitlab.com/ee/development/cicd/templates.html
# This specific template is located at:
# https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Diffblue-Cover.gitlab-ci.yml

variables:
  # Configure the following via the Diffblue Cover integration config for your project, or by
  # using CI/CD masked Variables.
  # For details, see https://docs-proxy.diffblue.io/features/cover-pipeline/cover-pipeline-for-gitlab

  # Diffblue Cover license key: DIFFBLUE_LICENSE_KEY
  # Refer to your welcome email or you can obtain a free trial key from
  # https://www.diffblue.com/try-cover/gitlab

  # GitLab access token: DIFFBLUE_ACCESS_TOKEN, DIFFBLUE_ACCESS_TOKEN_NAME
  # The access token should have a role of Developer or better and should have
  # api and write_repository permissions.

  # Diffblue Cover requires a minimum of 4GB of memory.
  JVM_ARGS: -Xmx4g

stages:
  - build

diffblue-cover:
  stage: build

  # Select the Cover CLI docker image to use with your CI tool.
  # Tag variations are produced for each supported JDK version.
  # Go to https://hub.docker.com/r/diffblue/cover-cli for details.
  # Note: To use the latest version of Diffblue Cover, use one of the latest-jdk<nn> tags.
  # To use a specific release version, use one of the yyyy.mm.dd-jdk<nn> tags.
  image: diffblue/cover-cli:latest-jdk17

  # Diffblue Cover currently only supports running on merge_request_events.
  rules:
    - if: $CI_PIPELINE_SOURCE == 'merge_request_event'

  # Diffblue Cover log files are saved to a .diffblue/ directory in the pipeline artifacts,
  # and are available for download once the pipeline completes.
  artifacts:
    paths:
      - "**/.diffblue/"

  script:

    # Diffblue Cover requires the project to be built before creating any tests.
    # Either specify the build command here (one of the following), or provide
    # prebuilt artifacts via a job dependency.

    # Maven project example (comment out the Gradle version if used):
    - mvn test-compile --batch-mode --no-transfer-progress

    # Gradle project example (comment out the Maven version if used):
    # - gradle testClasses

    # Diffblue Cover commands and options to run.
    #   dcover – the core Diffblue Cover command
    #   ci – enable the GitLab CI/CD integration via environment variables
    #   activate - activate the license key
    #   validate - remove non-compiling and failing tests
    #   create - create new tests for your project
    #   --maven – use the maven build tool
    # For detailed information on Cover CLI commands and options, see
    # https://docs-proxy.diffblue.io/features/cover-cli/commands-and-arguments
    - dcover
        ci
        activate
        validate --maven
        create --maven

    # Diffblue Cover will also respond to specific project labels:
    #   Diffblue Cover: Baseline
    #     Used to mark a merge request as requiring a full suite of tests to be written.
    #     This overrides the default behaviour where Cover will only write tests related
    #     to the code changes already in the merge request. This is useful when running Diffblue
    #     Cover for the first time on a project and when new product enhancements are released.
    #   Diffblue Cover: Skip
    #     Used to mark a merge request as requiring no tests to be written.

Masked variables

You can configure the Diffblue license key and associated GitLab access token using GitLab masked variables. Note that, if preferred, you can configure these properties directly, using the Diffblue Cover integration page on GitLab - see Integration config.

To add or update variables in the project or group settings (also see the Variables GitLab docs topic if needed):

1. In your group or project go to Settings > CI/CD and expand the Variables section.

2. Select Add variable and create each variable:

   • Key: Name of the variable. The three Diffblue variables to create are DIFFBLUE_LICENSE_KEY, DIFFBLUE_ACCESS_TOKEN_NAME, and DIFFBLUE_ACCESS_TOKEN. Note that offline licenses can also optionally be configured via environment variables, see below.

   • Value: Value for the variable:

     • DIFFBLUE_LICENSE_KEY - enter your Diffblue Cover License Key provided in your welcome email or by your organization.

     • DIFFBLUE_ACCESS_TOKEN_NAME, and DIFFBLUE_ACCESS_TOKEN - enter details of your GitLab access token (Name and Secret) to allow Diffblue Cover to access your project. In general, use a GitLab Project Access Token with a Developer role, plus api and write_repository permissions. If necessary you can use Group Access or Personal Access Tokens, again with api and write permissions. See the GitLab docs links below for full details on how to create an access token.

     

     • Optional: Note that if you have an offline license and wish to configure the offline license activation file this can be done by using the environment variable DIFFBLUE_OFFLINE_LICENSE_ACTIVATION_FILE_CONTENTS containing the contents of your offline license activation file ls_activation.lic

   • Type: Set to Variable.

   • Environment scope: Optional. All, or specific environments.

   • Protect variable Optional. If selected, the variable is only available in pipelines that run on protected branches or protected tags.

   • Mask variable Select this option. This will mask the variable’s Value in job logs.

After you create a variable, you can use it in your .gitlab-ci.yml file or in job scripts.

Log files

Diffblue Cover log files are saved to a .diffblue/ directory in the pipeline artifacts (as defined in the .gitlab-ci.yml file), and are available for download once the pipeline completes. These logs include user logs (also output during the run and visible in the job output) and support logs (if you ever require Diffblue support).

Here are some common issues that you may encounter when configuring and using Cover Pipeline for GitLab.

General

Diffblue Cover performs a number of checks before, during, and after creating unit tests. Cover can generate a range of output codes during these checks to provide general information and highlight any issues. These codes and associated messages are output during the run, visible in the job output, and saved to the log files. See Output Codes for details.

For general Diffblue Cover issues, see the Cover CLI Troubleshooting topic.

Diffblue license issues

License issues are reported using L - License Codes. Common issues include:

• L005 - The license has expired - Update your license key (see Configuration) and/or contact Diffblue Support for assistance.

• L009 - License key not found - Make sure you've entered the correct license key and in the format provided (see Configuration).

Access token issues

Access token issues are reported using the E120 - Git command failed output code (see E - Environment Codes). For example:

• Access token with incorrect permissions. In this case, check that the access token has correct permissions for the project (Developer role, plus api and write_repository scopes/permissions) - see Configuration.

INFO Pushing: 2023.11.02-RC2
ERROR E120: Git command failed
The following git command failed with an exit code of 128:
/usr/bin/git push origin HEAD:2023.11.02-RC2
Beginning of output
-------------------
> remote: You are not allowed to push code to this project.
> fatal: unable to access 'https://gitlab.com/diffblue-qa/DifferentialAnalysis.git/':
  The requested URL returned error: 403

• A mismatch of the access token name can lead to a Git error. GitLab requires the access token name used in the Cover Pipeline for GitLab integration to match the name of the access token in GitLab. Check and update your access token name as needed - see Configuration.

Diffblue Cover Docker issues

The Diffblue Cover docker image to use as part of your project pipeline is specified in your .gitlab-ci.yml file (see Configuration). Tag variations are provided for each supported JDK version (see https://hub.docker.com/r/diffblue/cover-cli for details). An error in the naming of the docker image will cause a failure to download the docker file:

To resolve this issue, simply update the docker image name in your .gitlab-ci.yml file.

No unit test changes

If no tests are created, updated, or removed as part of your merge request, you may have selected the Diffblue Cover: Skip label in the merge request. This label is used to mark a merge request as requiring no tests to be written. See Labels.

Once you're familiar with the topics covered under Cover Pipeline for GitLab, check out the following to help extend your unit test capabilities even further:

• Specs & Reqs Specifications, prerequisites, and general project requirements are all covered in Specs & Reqs.

• Cover Reports Integration Additional coverage-reports and upload commands can be added to your pipeline in order to integrate Diffblue Cover: Baseline labelled CI runs into Cover Reports- see Generate and upload reports bundlesfor details.

• dcover Commands in a pipeline Additional optional arguments can be used with the create and validate commands listed in your .gitlab-ci.yml file for your project - see Commands & Arguments for details.

• Output Codes Diffblue Cover performs a number of checks before, during, and after creating unit tests. Cover can generate a range of output codes during these checks to provide general information and highlight any issues. These codes and associated messages are output during the run, visible in the job output, and saved to the log files. See Output Codes for details.

• Test Coverage See the How to improve code coverage topic for general information on ways to improve your overall coverage across your Java projects.

• Developer Tools Diffblue Cover is also available as two developer tools - Cover Plugin for IntelliJ (write tests directly within the IntelliJ IDE) and Cover CLI (full command line tool) - both of which are bundled with Cover Pipeline for GitLab. These tools can be used by your development staff to automatically write tests during the development stages of a project, thereby allowing them to check for regressions and unplanned code behavior changes even before the code is committed back to GitLab. See the Free trial or Get started topics to check out the tools for yourself.