Mercurial > hg > Blog

annotate content/Maven/sonar-vs-jacoco-vs-surefire.md @ 103:6aa2fa328a94

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
New blog post on sonar and maven
author Dirk Olmes <dirk.olmes@codedo.de>
date Tue, 12 May 2020 08:30:22 +0200
parents
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
103
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
1 Title: Generating test coverage for Sonar with maven, surefire and jacoco
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
2 Date: 2020-05-11
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
3 Lang: en
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
4
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
5 At work we're using [Sonar](https://www.sonarqube.org) to keep our code quality under control. One integral part of code quality is test coverage and Sonar offers coverage metrics in the UI. However, the [Sonar docs on code coverage](https://docs.sonarqube.org/latest/analysis/coverage/) are a bit sparse (at best) and don't tell you the exact steps to run for getting coverage with a Maven build.
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
6
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
7 With a bit of googling I was eventually able to find the correct steps to get coverage. The key to success is the [Jacoco Maven plugin](https://www.eclemma.org/jacoco/trunk/doc/maven.html). In a first Maven execution we let jacoco record coverage statistics while running the unit tests:
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
8
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
9 :::bash
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
10 mvn clean org.jacoco:jacoco-maven-plugin:prepare-agent install
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
11
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
12 In the build log you see the jacoco-maven-plugin in action (we'll get back to this in a while):
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
13
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
14 [INFO] --- jacoco-maven-plugin:0.8.5:prepare-agent (default-cli) @ sonar-code-coverage ---
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
15 [INFO] argLine set to -javaagent:/.m2/repository/org/jacoco/org.jacoco.agent/0.8.5/org.jacoco.agent-0.8.5-runtime.jar=destfile=/sonar-code-coverage/target/jacoco.exec
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
16
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
17 Then in a second Maven execution we'll pick up those coverage statistics and convert them to XML so that Sonar can include them in its analysis:
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
18
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
19 :::bash
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
20 mvn org.jacoco:jacoco-maven-plugin:report org.sonarsource.scanner.maven:sonar-maven-plugin:sonar
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
21
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
22 **Hooray! Coverage info in Sonar.**
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
23
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
24 The trouble starts when you need to pass custom arguments through surefire down to the JVM that will run the tests. This is done through the `<argLine>` configuration parameter of maven-surefire-plugin:
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
25
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
26 :::xml
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
27 <plugin>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
28 <groupId>org.apache.maven.plugins</groupId>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
29 <artifactId>maven-surefire-plugin</artifactId>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
30 <configuration>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
31 <argLine>-Xmx128m</argLine>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
32 </configuration>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
33 </plugin>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
34
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
35 Looking back at the output of jacoco-maven-plugin above you'll notice that we're setting exactly the same `<argLine>` parameter as jacoco-maven-plugin does. If you don't take extra steps only one configuration can win - and that's the configuration you put into your pom. You have to run `mvn -X` to actually see this, though:
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
36
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
37 [DEBUG] Forking command line: /bin/sh -c cd /sonar-code-coverage && /opt/openjdk-bin-8.252_p09/jre/bin/java -Xmx128m -jar /sonar-code-coverage/target/surefire/surefirebooter9099169912642065145.jar /sonar-code-coverage/target/surefire 2020-05-12T07-39-04_225-jvmRun1 surefire3709014631518520641tmp surefire_01706858247537265191tmp
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
38
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
39 Note how the `-javaagent:...` argLine is not included in the command line arguments of the JVM that's forked to run the tests.
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
40
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
41 A way to fix this is through creative use of Maven build properties. Configure the jacoco-maven-plugin to set a different property than `argLine`:
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
42
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
43 :::xml
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
44 <plugin>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
45 <groupId>org.jacoco</groupId>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
46 <artifactId>jacoco-maven-plugin</artifactId>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
47 <configuration>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
48 <propertyName>surefire.argLine.sonar</propertyName>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
49 </configuration>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
50 </plugin>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
51
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
52 Next, configure maven-surefire-plugin to include that property into its `<argLine>`:
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
53
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
54 :::xml
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
55 <plugin>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
56 <groupId>org.apache.maven.plugins</groupId>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
57 <artifactId>maven-surefire-plugin</artifactId>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
58 <configuration>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
59 <argLine>-Xmx128m @{surefire.argLine.sonar}</argLine>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
60 </configuration>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
61 </plugin>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
62
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
63 Note that you have to use [late evaluation](http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#argLine) of the property because it's not present when the Maven reactor starts but only after jacoco-maven-plugin has run.
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
64
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
65 Just one last hack is required to make the build work even if you don't run jacoco coverage as part of your build. Specify an empty property `surefire.argLine.sonar` in your project's properties so that it always exists and can be expanded.
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
66
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
67 For further reference I've put [a demo project up on github](https://github.com/dirk-olmes/sonar-code-coverage).