Difference between revisions of "Sonar + maven configuration + Jenkins"
Line 300: Line 300:
 
==Run tests in specific order==
 
==Run tests in specific order==
  
−
If for some reasons you need to run your tests in a particular order you can say so in the POM:
+
If for some reasons you need to run your tests in a particular order you can say do so in Maven and/or Java
 +
 
 +
'''Reminder''' : if you have to run your tests in a particular order then you should refactor your tests! These tricks should not let you escape from that!
 +
 
 +
 
 +
===Maven===
 +
 
 +
The <code>runOrder</code> is a nice trick.
  
 
<syntaxhighlight lang="xml">
 
<syntaxhighlight lang="xml">
Line 324: Line 331:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
−
The <code>runOrder</code> is a nice trick.
 
  
 +
===Java===
  
−
'''Warning''' : if you have to run your tests in a particular order then you should refactor your tests! This trick should not let you escape from that!
+
Since jUnit 4.11 you can set the order of your tests.
  
 +
* Set order to the Class level: https://github.com/junit-team/junit/wiki/Test-execution-order
 +
* Set custom order: http://memorynotfound.com/run-junit-tests-method-order/
  
  

Revision as of 17:01, 20 May 2015


SonarQube requires quite some configuration to be fully useful!

This article explains how to configure your maven projects to use SonarQube properly, bringing both Unit and Integration tests coverage.


Principle

To work well SonarQube requires 4 configuration steps:

TODO: nice picture


Key points:

  • SONAR is configured for JAVA language and it will use Jacoco as coverage tool.
  • Each maven module will have its own Unit Tests results (Surefire reports + .exec file) inside its own target directory
  • The Integration Tests results are common. Meaning, all modules will write in the same directory. That directory is relative to the maven execution.


Maven configuration

Sonar properties

First of all you need to configure SONAR by setting up some properties and paths.

Technical details:

  • Surefire unit tests plugin
    • Outputs in ${project.build.directory}
    • 1 XML file per test class in: ${sonar.junit.reportsPath}
    • SONAR report file (jacoco-ut.exec) will be available at ${sonar.out.unitTestsReport}
  • Failsafe integration tests plugin
    • All outputs in the same directory, relative to the local execution: ${session.executionRootDirectory}


POM.xml extract: 

    <properties>
        ...

        <!-- Maven plugins -->
        <maven-surefire-plugin.version>2.18.1</maven-surefire-plugin.version>
        <maven-failsafe-plugin.version>2.18.1</maven-failsafe-plugin.version>
 
        <!-- Test frameworks -->
        <junit.version>4.12</junit.version>
 
        <!-- ==== SONARQUBE quality metrics ==== -->
        <maven.jacoco.version>0.7.4.201502262128</maven.jacoco.version>
        <!-- Global Sonar settings. Do not change them! -->
        <sonar.language>java</sonar.language>
        <sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>
        <sonar.core.codeCoveragePlugin>jacoco</sonar.core.codeCoveragePlugin>
        <jacoco.lib.path>
            ${settings.localRepository}/org/jacoco/org.jacoco.agent/${maven.jacoco.version}/org.jacoco.agent-${maven.jacoco.version}-runtime.jar
        </jacoco.lib.path>
        <javaagent>${jacoco.lib.path}</javaagent>
        <!-- Don't let Sonar execute tests. We will let Maven do it during build phase -->
        <sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
        <!-- Report -->
        <!-- IMPORTANT: integration test path must be ABSOLUTE, especially for muli-modules projects -->
        <!--            ${session.executionRootDirectory} = directory from where the "mvn" command is run -->
        <!-- a) Where sonar will find the standard test reports -->
        <sonar.surefire.reportsPath>${project.build.directory}/surefire-reports</sonar.surefire.reportsPath>
        <sonar.failsafe.reportsPath>${session.executionRootDirectory}/target/failsafe-reports</sonar.failsafe.reportsPath>
        <!-- b) Sonar specific reports -->
        <sonar.jacoco.reportPath>${project.build.directory}/jacoco-unit.exec</sonar.jacoco.reportPath>
        <sonar.jacoco.itReportPath>${session.executionRootDirectory}/target/reports/jacoco-it.exec</sonar.jacoco.itReportPath>
    </properties>


Jacoco plugin (coverage agent)

This is how you should configure this plugin: 

    <build>
        <plugins>
            <!-- == Jacoco agent configuration == -->
            <!-- This will auto-generate the right jacoco command for Unit | Integration tests -->
            <plugin>
                <groupId>org.jacoco</groupId>
                <artifactId>jacoco-maven-plugin</artifactId>
                <version>${maven.jacoco.version}</version>
                <executions> 
                    <!-- Unit tests configuration -->
                    <execution>
                        <id>prepare-unit-test-agent</id>
                        <phase>process-test-classes</phase>
                        <goals>
                            <goal>prepare-agent</goal>
                        </goals>
                        <configuration>
                            <destFile>${sonar.jacoco.reportPath}</destFile>
                            <propertyName>jacoco.agent.ut.arg</propertyName>
                            <append>true</append>
                        </configuration>
                    </execution>
                    <!-- Integration tests configuration -->
                    <execution>
                        <id>prepare-integration-test-agent</id>
                        <phase>pre-integration-test</phase>
                        <goals>
                            <goal>prepare-agent</goal>
                        </goals>
                        <configuration>
                            <destFile>${sonar.jacoco.itReportPath}</destFile> 
                            <propertyName>jacoco.agent.it.arg</propertyName>
                            <append>true</append>                 
                        </configuration>
                    </execution>
                </executions>
            </plugin>

            <plugin> Surefire (UT) </plugin>
            <plugin> Failsafe (IT) </plugin>

        </plugins>
    </build>


This will defined 2 properties, depending on the maven phase:

Phase ArgLine Value
Unit Tests process-test-classes jacoco.agent.ut.arg -javaagent:${jacoco.lib.path}=destFile=${sonar.jacoco.reportPath},append=true
Integration Tests pre-integration-test jacoco.agent.it.arg -javaagent:${jacoco.lib.path}=destFile=${sonar.jacoco.itReportPath},append=true

These properties will be re-use in the corresponding maven plugin.


Surefire (UT tests)

Surefire will process the Unit Tests.

!! Do not use Surefire for your integration tests because Surefire produce different outputs per maven module !! Integration tests requires to use 1 output for all ! 

    <build>
        <plugins>

            <plugin> Maven-jacoco </plugin>

            <!-- == process UNIT tests == -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>${maven-surefire-plugin.version}</version>
                <configuration>
                    <!-- Do not run if no tests -->
                    <skipTests>${skipTests}</skipTests>
                    <!-- a) Surefire must create junit reports where sonar can find it later during analysis --> 
                    <reportsDirectory>${sonar.jacoco.reportPath}</reportsDirectory>
                    <!-- b) Jacoco specific settings (command auto-generate by Sonar plugin) -->
                    <argLine>${jacoco.agent.ut.arg}</argLine>
                    <!-- Exclude integration tests -->
                    <excludes>
                        <exclude>**/*IT.java</exclude>
                        <exclude>**/*IntegrationTest.java</exclude>
                        <exclude>**/*Gwt.java</exclude>
                    </excludes>
                </configuration>
            </plugin>

            <plugin> Failsafe (IT) </plugin>

        </plugins>
    </build>


Notes:

  • No tests will be run if mvn clean install -DskipTests
  • Each module has its own surefire-reports + Sonar report
  • The exclusion list is something that depends on your own situation


Failsafe configuration (IT tests)

Failsafe will process the Integration Tests.

!! Do not use Surefire for your integration tests because Surefire produce different outputs per maven module !! Integration tests requires to use 1 output for all ! 

    <build>
        <plugins>

            <plugin> Maven-jacoco </plugin>
            <plugin> Surefire (UT) </plugin>

            <!-- == process INTEGRATION tests == -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-failsafe-plugin</artifactId>
                <version>${maven-failsafe-plugin.version}</version>
                <configuration>
                    <!-- Do not run if no tests -->
                    <skipTests>${skipIntegrationTests}</skipTests>
                    <!-- a) Failsafe must create junit reports where sonar can find it later during analysis -->
                    <reportsDirectory>${sonar.failsafe.reportsPath}</reportsDirectory>
                    <!-- JVM settings -->
                    <argLine>-Xmx1024m -XX:maxPermSize:256m</argLine>
                    <forkCount>1</forkCount>
                    <reuseForks>true</reuseForks>
                    <!-- <runOrder>alphabetical</runOrder> -->
                    <!-- Sonar specific settings (command auto-generate by Sonar plugin) -->                  
                    <argLine>${jacoco.agent.it.arg}</argLine>
                    <includes>
                        <include>**/*IntegrationTest.java</include>
                        <include>**/*IT.java</include>
                    </includes>
                    <excludes>
                        <exclude>**/*Gwt.java</exclude>
                    </excludes>
                </configuration>
                <!-- Only run failsafe when required -->
                <executions>
                    <execution>
                        <id>integration-test</id>
                        <phase>integration-test</phase>
                        <goals>
                            <goal>integration-test</goal>
                            <goal>verify</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

        </plugins>
    </build>


Notes:

  • No tests will be run if mvn clean install -DskipIntegrationTests
  • Only 1 report for all modules, that depends on the ${session.executionRootDirectory}
  • The exclusion | inclusion list is something that depends on your own situation
  • The JVM settings can be arranged


Jenkins configuration

Now that you can generate these reports, especially the integration results, you need to configure Jenkins to use them!

Jenkins SonarQube configuration


To set up the Sonar Jenkins plugin to process already generated JaCoCo data files, the following properties must be specified in the "Additional properties" text box, under the Sonar installation being configured:

Header text Header text
sonar.dynamicAnalysis=reuseReports Tells SonarQube to reuse existing reports for unit tests execution and coverage reports
-Dsonar.jacoco.reportPath=target/reports/jacoco-unit.exec (Optional) This should be automatic

Path to Unit Test SONAR report.

-Dsonar.surefire.reportsPath=target/surefire-reports (Optional) This should be automatic

Tell Sonar where to find Surefire per class / per method reports

-Dsonar.jacoco.itReportPath={ABSOLUTE_PATH_TO_JENKINS_JOB}/workspace/target/reports/jacoco-it.exec Absolute path to the integration tests.



Real life example

You can check out one of my GitHub project:


Even better, you can check out this excellent tutorial:


Appendices

Run tests in specific order

If for some reasons you need to run your tests in a particular order you can say do so in Maven and/or Java

Reminder : if you have to run your tests in a particular order then you should refactor your tests! These tricks should not let you escape from that!


Maven

The runOrder is a nice trick. 

    <build>
        <plugins>
            <plugin>
                SUREFIRE or FAILSAFE

                <configuration>
                    <!-- JVM settings -->
                    <argLine>-Xmx1024m -XX:maxPermSize:256m</argLine>
                    <forkCount>1</forkCount>
                    <reuseForks>true</reuseForks>
                    <runOrder>alphabetical</runOrder>
                    <!-- Jacoco execution -->                 
                    <argLine>${jacoco.agent.it.arg}</argLine>
                    ...
                </configuration>
            </plugin>

        </plugins>
    </build>


Java

Since jUnit 4.11 you can set the order of your tests.


References

This article is based on my daily work in VEHCO.

To update to SonarQube I used the following articles and code examples:

Jenkins configuration:

Retrieved from "http://www.daxiongmao.eu/wiki/index.php?title=Sonar_%2B_maven_configuration_%2B_Jenkins&oldid=2362"