Sonar + maven principle

SonarQube requires quite some configuration to be fully useful!

This article explains the key principles + what is UNIT and INTEGRATION tests from my point of view.

Principle

What are Unit / Integration tests ?

By default "code coverage" only consider the tests from the same projects.

Let's take an example to make things easier...

Consider a project with 4 modules such as:

Then consider the following inter-dependencies:

Definition:

• Unit Tests coverage (UT) = code coverage for each module only. == How much tests of 'module A' test 'module A' ??
• Integration Tests coverage (IT) = overall code coverage. == Once global compilation and all the tests have been run, how much of 'module A' was used and covered ?

... In our example:

• For "Module A" only the jUnit in "Module A" are computed in unit test coverage
• The coverage provided by the Module C "tests" is NOT taken into account by default! You need to enable and configure the integration test coverage.

... Why ?!? This limitation comes from the default SONAR behaviour. By default it creates 1 coverage report per component (maven project). To get the overall coverage you need a common report that each project can improve and use.

Process overview

To work well SonarQube requires a bit of Maven configuration + Jenkins build adjustment:

• Maven specific properties
• Maven build plug-ins
  • Maven-jacoco (code coverage tool)
  • Maven-surefire (unit tests reports)
  • Maven-failsafe (integration tests reports)
• Maven jacoco dependency

Key points:

• SONAR is configured for JAVA language and it will use Jacoco as coverage tool.
• Each Maven module will have its own Unit Tests results (Surefire reports + .exec file) inside its own target directory
• The Integration Tests results are common. Meaning, all modules will write in the same directory. That directory is relative to the maven execution.

Real life example

You can check out one of my GitHub project:

• https://github.com/guihome-diaz/Languages > pom.xml

Even better, you can check out this excellent tutorial:

• https://github.com/dgageot/coverage

Appendices

Run tests in specific order

If for some reasons you need to run your tests in a particular order you can say do so in Maven and/or Java

Reminder : if you have to run your tests in a particular order then you should refactor your tests! These tricks should not let you escape from that!

Maven

The runOrder is a nice trick.

    <build>
        <plugins>
            <plugin>
                SUREFIRE or FAILSAFE

                <configuration>
                    <!-- JVM settings -->
                    <argLine>-Xmx1024m -XX:maxPermSize:256m</argLine>
                    <forkCount>1</forkCount>
                    <reuseForks>true</reuseForks>
                    <runOrder>alphabetical</runOrder>
                    <!-- Jacoco execution -->                 
                    <argLine>${jacoco.agent.it.arg}</argLine>
                    ...
                </configuration>
            </plugin>

        </plugins>
    </build>

Java

Since jUnit 4.11 you can set the order of your tests.

• Set order to the Class level: https://github.com/junit-team/junit/wiki/Test-execution-order
• Set custom order: http://memorynotfound.com/run-junit-tests-method-order/

References

This article is based on my daily work in VEHCO.

To update to SonarQube I used the following articles and code examples:

• http://www.aheritier.net/maven-failsafe-sonar-and-jacoco-are-in-a-boat/
• VERY good GitHub example, provided by the same author: https://github.com/dgageot/coverage
• http://stackoverflow.com/questions/26253277/jacoco-agent-for-integration-tests-on-remote-machine

Jenkins configuration:

• http://blog.ccbill.com/2014/07/code-coverage-with-surefire-and-jacoco.html