This article explains how to configure your maven projects to produce reports + how to configure Jenkins with SonarQube, bringing both Unit and Integration tests coverage.

1 Requirements
2 Maven / Sonar key points
3 Maven properties
4 Jacoco plugin (coverage agent)
5 Surefire (Unit tests)
6 Failsafe (Integration tests)
7 Maven dependencies
8 Maven profile
9 Jenkins configuration
10 Real life example
11 Appendices
- 11.1 Run tests in specific order
  - 11.1.1 Maven
  - 11.1.2 Java
12 References

Requirements

Maven must run on Java 7 or +
You must use Sonarqube 5.3 or +

Maven / Sonar key points

Maven key properties

Here are some properties you must know and use to configure Sonar:

${project.reporting.outputDirectory} == module/target/site/
${project.build.directory} == module/target/
${session.executionRootDirectory} == root folder where mvn clean install is launched
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>

Sonar properties

First of all you need to configure SONAR by setting up some properties and paths, see Surefire and Failsafe below.

To optimize SONAR execution we will set the following behaviour:

Language == Java
Coverage tool == JaCoCo
Behaviour == re-use Surefire and Failsafe reports. Do NOT re-compile for Sonar.

Surefire Unit Tests

Results are saved in each Maven module:

Results are saved in XML format in $module/target/site/surefire-reports
Results are saved in HTML format in $module/target/site/jacoco-ut. You can open index.html to browse the results.
Results are saved in Binary (jacoco format) in $module/target/jacoco.exec

Failsafe Integration tests

Only XML and HTML results are saved in each Maven module. Consolidate results are saved in the execution path ./target

Results are saved in XML format in $module/target/site/failsafe-reports
Results are saved in HTML format in $module/target/site/jacoco-it. You can open index.html to browse the results.
Results are saved in Binary (jacoco format) in $maven_execution_place/target/jacoco-it.exec

Maven properties

POM.xml extract:

    <properties>
                <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>    
                <junit.version>4.12</junit.version>

                <!-- ================================================================= -->
                <!-- ==== FAILSAFE (IT tests) + SUREFIRE (Unit tests) + SONARQUBE ==== -->
                <!-- ================================================================= -->
                <maven-surefire-plugin.version>2.18.1</maven-surefire-plugin.version>
                <maven-failsafe-plugin.version>2.18.1</maven-failsafe-plugin.version>
                <maven.jacoco.version>0.7.7.201606060606</maven.jacoco.version>        
                
                <!-- ========================================== -->
                <!-- Global Sonar settings. Do not change them! -->
                <!-- ========================================== -->
                <sonar.language>java</sonar.language>
                <sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>
                <sonar.core.codeCoveragePlugin>jacoco</sonar.core.codeCoveragePlugin>
                <jacoco.lib.path>
                    ${settings.localRepository}/org/jacoco/org.jacoco.agent/${maven.jacoco.version}/org.jacoco.agent-${maven.jacoco.version}-runtime.jar
                </jacoco.lib.path>
                <javaagent>${jacoco.lib.path}</javaagent>
                <!-- Don't let Sonar execute tests! Maven will run them and produce some reports for Sonar -->
                <sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
                
                <!-- =============== -->
                <!-- Quality reports -->
                <!-- =============== -->
                <!-- XML reports (go to $/target/site/*-reports) -->
                <sonar.surefire.reportsPath>${project.reporting.outputDirectory}/surefire-reports</sonar.surefire.reportsPath>
                <sonar.failsafe.reportsPath>${project.reporting.outputDirectory}/failsafe-reports</sonar.failsafe.reportsPath>
                <!-- Web reports (= go to $/target/site/jacoco-*/index.html to browse results) -->
                <jacoco.ut.summary>${project.reporting.outputDirectory}/jacoco-ut</jacoco.ut.summary>
                <jacoco.it.summary>${project.reporting.outputDirectory}/jacoco-it</jacoco.it.summary>
                <!-- Binary format [= Sonar format] -->
                <!--    1) Unit tests reports must be in each module $/target/ project -->
                <jacoco.ut.execution.data.file>${project.build.directory}/jacoco.exec</jacoco.ut.execution.data.file>
                <!--    2) Integration tests reports must be consolidated together. If not then you will probably lost multi-project coverage -->
                <jacoco.it.execution.data.file>${session.executionRootDirectory}/target/jacoco-it.exec</jacoco.it.execution.data.file>
            </properties>

Jacoco plugin (coverage agent)

This is how you should configure this plugin:

    <build>
        <plugins>
                    <!-- #################### -->
                    <!-- SONAR # CODE METRICS -->
                    <!-- #################### -->
                    <!-- This will auto-generate the right jacoco command for Unit | Integration tests -->
                    <plugin>
                        <groupId>org.jacoco</groupId>
                        <artifactId>jacoco-maven-plugin</artifactId>
                        <version>${maven.jacoco.version}</version>
                        <executions>
                        
                            <!-- #- - - - - - - - - - - - # -->
                            <!-- Unit tests configuration -->
                            <!-- #- - - - - - - - - - - - # -->           
                            <!-- JaCoCo runtime agent which is passed as VM argument when Maven Surefire plugin is executed. -->
                            <execution>
                                <id>pre-unit-test</id>                  
                                <goals>
                                    <goal>prepare-agent</goal>
                                </goals>
                                <configuration>
                                    <!-- Surefire (Unit tests) reports will be saved as XML in... -->
                                    <destFile>${jacoco.ut.execution.data.file}</destFile>
                                    <propertyName>surefireArgLine</propertyName>
                                    <append>true</append>
                                </configuration>
                            </execution>
                             <!-- Ensures code coverage report for unit tests is created after unit tests have been run. -->
                            <execution>
                                <id>post-unit-test</id>
                                <phase>test</phase>
                                <goals>
                                    <goal>report</goal>
                                </goals>
                                <configuration>
                                    <!-- Surefire (Unit tests) reports will be saved as Binary in... -->
                                    <dataFile>${jacoco.ut.execution.data.file}</dataFile>
                                    <outputDirectory>${jacoco.ut.summary}</outputDirectory>
                                    <append>true</append>
                                </configuration>
                            </execution>
                            
                            <!-- #- - - - - - - - - - - - # -->
                            <!-- Integration tests configuration -->
                            <!-- #- - - - - - - - - - - - # -->
                            <!-- JaCoCo runtime agent which is passed as VM argument when Maven Failsafe plugin is executed. -->                   
                            <execution>
                                <id>pre-integration-test</id>
                                <phase>pre-integration-test</phase>
                                <goals>
                                    <goal>prepare-agent</goal>
                                </goals>
                                <configuration>
                                    <!-- Failsafe (Integration tests) reports will be saved as XML in... -->
                                    <destFile>${jacoco.it.execution.data.file}</destFile>
                                    <propertyName>failsafeArgLine</propertyName>                            
                                    <append>true</append>                 
                                </configuration>
                            </execution>
                             <!-- Ensures code coverage report for integration tests is created after integration tests have been run. -->
                            <execution>
                                <id>post-integration-test</id>
                                <phase>post-integration-test</phase>
                                <goals>
                                    <goal>report</goal>
                                </goals>
                                <configuration>
                                    <!-- Failsafe (Integration tests) reports will be saved as Binary in... -->
                                    <dataFile>${jacoco.it.execution.data.file}</dataFile>
                                    <outputDirectory>${jacoco.it.summary}</outputDirectory>                         
                                    <append>true</append>                 
                                </configuration>
                            </execution>
                        </executions>
                    </plugin>

            <plugin> Surefire (UT) </plugin>
            <plugin> Failsafe (IT) </plugin>

        </plugins>
    </build>

This configuration force the generation of both XML and Binary reports.

Surefire (Unit tests)

Surefire will process the Unit Tests.

!! Do not use Surefire for your integration tests because Surefire produce different outputs per maven module !!

    <build>
        <plugins>

            <plugin> Maven-jacoco </plugin>

                    <!-- == process UNIT tests == -->
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-surefire-plugin</artifactId>
                        <version>${maven-surefire-plugin.version}</version>
                        <configuration>                
                            <!-- Sets the VM argument line used when unit tests are run. -->
                            <argLine>-Xms256m -Xmx1G -XX:PermSize=256m -XX:MaxPermSize=512m ${surefireArgLine}</argLine>
                            <!-- Skip unit tests if asked -->
                            <skipTests>${skipTests}</skipTests>
                            <!-- Excludes integration tests when unit tests are run. -->
                            <excludes>
                                <exclude>**/*IT.java</exclude>
                                <exclude>**/*IntegrationTest.java</exclude>
                                <exclude>**/*GWT*.java</exclude>
                            </excludes>
                            <!-- Save reports in particular directory -->
                            <reportsDirectory>${sonar.surefire.reportsPath}</reportsDirectory>
                        </configuration>
                    </plugin>

            <plugin> Failsafe (IT) </plugin>

        </plugins>
    </build>

Notes:

No tests will be run if mvn clean install -DskipTests
The exclusion list is something that depends on your own situation

Failsafe (Integration tests)

Failsafe will process the Integration Tests.

    <build>
        <plugins>

            <plugin> Maven-jacoco </plugin>
            <plugin> Surefire (UT) </plugin>

                    <!-- == process INTEGRATION tests == -->
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-failsafe-plugin</artifactId>
                        <version>${maven-failsafe-plugin.version}</version>              
                        <configuration>                        
                            <!-- Sets the VM argument line used when integration tests are run. -->
                            <argLine>-Xms256m -Xmx1G -XX:PermSize=256m -XX:MaxPermSize=512m ${failsafeArgLine}</argLine>
                            <!-- Skip integration tests if asked -->
                            <skipTests>${skipIntegrationTests}</skipTests>
                            <includes>
                                <include>**/*IntegrationTest.java</include>
                                <include>**/*IT.java</include>                         
                            </includes>      
                            <!-- Excludes GWT crap -->
                            <excludes>
                                <exclude>**/*GWT*.java</exclude>
                            </excludes>                   
                            <!-- Save reports in particular directory -->
                            <reportsDirectory>${sonar.failsafe.reportsPath}</reportsDirectory>
                        </configuration>
                        <!-- Only run failsafe when required -->
                        <executions>
                            <execution>
                                <id>integration-tests</id>
                                <goals>
                                    <goal>integration-test</goal>
                                    <goal>verify</goal>
                                </goals>
                            </execution>
                        </executions>
                    </plugin> 

        </plugins>
    </build>

Notes:

No tests will be run if mvn clean install -DskipIntegrationTests
The exclusion | inclusion list is something that depends on your own situation

Maven dependencies

At last, you need to add the jacoco dependency:

(i) 2016-07-19 # this step is NOT required anymore on latest Jenkins & Sonarqube

    <dependencies>
        <!-- jUnit -->
        <dependency>
             <groupId>junit</groupId>
             <artifactId>junit</artifactId>
             <version>${junit.version}</version>
             <scope>test</scope>
        </dependency>      

        <!-- SonarQube dependencies -->
        <dependency>
            <groupId>org.jacoco</groupId>
            <artifactId>org.jacoco.agent</artifactId>
            <version>${maven.jacoco.version}</version>
            <classifier>runtime</classifier>
            <scope>test</scope>
        </dependency>
    </dependencies>

Maven profile

Put all the following elements inside a Maven profile:

        <profile>
            <id>sonar</id>
            <activation>
                    <activeByDefault>false</activeByDefault>
            </activation>

            <properties> ... </properties>
            <build> <plugins> ... </plugins> </build>
            <dependencies> ... </dependencies>
        </profile>
    </profiles>

To use it:

Standard use = mvn clean install -P sonar
Skip Unit tests = mvn clean install -DskipTests -P sonar
Skip Integration tests = mvn clean install -DskipIntegrationTests -P sonar

Jenkins configuration

Now that you can generate these reports, especially the integration results, you need to configure Jenkins to use them!

To set up the Sonar Jenkins plugin to process already generated JaCoCo data files, the following properties must be specified in the "Additional properties" text box, under the Sonar installation being configured:

-Dsonar.dynamicAnalysis=reuseReports ==> Tells SonarQube to reuse existing reports for unit tests execution and coverage reports
-Dsonar.jacoco.itReportPath=${ABSOLUTE_PATH_TO_JENKINS}/workspace/${JENKINS_JOB}/target/reports/jacoco-it.exec => integration tests report to process

Real life example

You can download a multi-project POM example: http://www.daxiongmao.eu/wiki_upload_files/code/sonar/pom.xml

You can check out these excellent tutorials:

Appendices

Run tests in specific order

If for some reasons you need to run your tests in a particular order you can say do so in Maven and/or Java

Reminder : if you have to run your tests in a particular order then you should refactor your tests! These tricks should not let you escape from that!

Maven

The runOrder is a nice trick.

    <build>
        <plugins>
            <plugin>
                SUREFIRE or FAILSAFE

                <configuration>
                    <!-- JVM settings -->
                    <argLine>-Xmx1024m -XX:maxPermSize:256m</argLine>
                    <forkCount>1</forkCount>
                    <reuseForks>true</reuseForks>
                    <runOrder>alphabetical</runOrder>
                    <!-- Jacoco execution -->                 
                    <argLine>${jacoco.agent.it.arg}</argLine>
                    ...
                </configuration>
            </plugin>

        </plugins>
    </build>

Java

Since jUnit 4.11 you can set the order of your tests.

Set order to the Class level: https://github.com/junit-team/junit/wiki/Test-execution-order
Set custom order: http://memorynotfound.com/run-junit-tests-method-order/

References

This article is based on my daily work in VEHCO.

To update to SonarQube I used the following articles and code examples:

http://www.aheritier.net/maven-failsafe-sonar-and-jacoco-are-in-a-boat/
VERY good GitHub example, provided by the same author: https://github.com/dgageot/coverage
http://stackoverflow.com/questions/26253277/jacoco-agent-for-integration-tests-on-remote-machine

Jenkins configuration:

http://blog.ccbill.com/2014/07/code-coverage-with-surefire-and-jacoco.html

Anonymous

Search

Navigation

Navigation

Categories

Wiki tools

Wiki tools

Sonar + maven configuration + Jenkins

Namespaces

Page actions

Contents

Requirements

Maven / Sonar key points

Maven key properties

Sonar properties

Surefire Unit Tests

Failsafe Integration tests

Maven properties

Jacoco plugin (coverage agent)

Surefire (Unit tests)

Failsafe (Integration tests)

Maven dependencies

Maven profile

Jenkins configuration

Real life example

Appendices

Run tests in specific order

Maven

Java

References

Anonymous

Search

Navigation

Wiki tools

Page tools

Categories

Sonar + maven configuration + Jenkins

Contents

Requirements

Maven / Sonar key points

Maven key properties

Sonar properties

Surefire Unit Tests

Failsafe Integration tests

Maven properties

Jacoco plugin (coverage agent)

Surefire (Unit tests)

Failsafe (Integration tests)

Maven dependencies

Maven profile

Jenkins configuration

Real life example

Appendices

Run tests in specific order

Maven

Java

References