Mercurial > hg > Blog

annotate content/Maven/sonar-vs-jacoco-vs-surefire.md @ 112:cf31bf5fce72 default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Author of the blog post as mail header for efficient spam filtering
author Dirk Olmes <dirk.olmes@codedo.de>
date Tue, 06 Sep 2022 07:04:11 +0200
parents 6aa2fa328a94
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
103
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
1 Title: Generating test coverage for Sonar with maven, surefire and jacoco
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
2 Date: 2020-05-11
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
3 Lang: en
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
4
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
5 At work we're using [Sonar](https://www.sonarqube.org) to keep our code quality under control. One integral part of code quality is test coverage and Sonar offers coverage metrics in the UI. However, the [Sonar docs on code coverage](https://docs.sonarqube.org/latest/analysis/coverage/) are a bit sparse (at best) and don't tell you the exact steps to run for getting coverage with a Maven build.
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
6
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
7 With a bit of googling I was eventually able to find the correct steps to get coverage. The key to success is the [Jacoco Maven plugin](https://www.eclemma.org/jacoco/trunk/doc/maven.html). In a first Maven execution we let jacoco record coverage statistics while running the unit tests:
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
8
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
9 :::bash
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
10 mvn clean org.jacoco:jacoco-maven-plugin:prepare-agent install
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
11
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
12 In the build log you see the jacoco-maven-plugin in action (we'll get back to this in a while):
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
13
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
14 [INFO] --- jacoco-maven-plugin:0.8.5:prepare-agent (default-cli) @ sonar-code-coverage ---
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
15 [INFO] argLine set to -javaagent:/.m2/repository/org/jacoco/org.jacoco.agent/0.8.5/org.jacoco.agent-0.8.5-runtime.jar=destfile=/sonar-code-coverage/target/jacoco.exec
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
16
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
17 Then in a second Maven execution we'll pick up those coverage statistics and convert them to XML so that Sonar can include them in its analysis:
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
18
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
19 :::bash
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
20 mvn org.jacoco:jacoco-maven-plugin:report org.sonarsource.scanner.maven:sonar-maven-plugin:sonar
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
21
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
22 **Hooray! Coverage info in Sonar.**
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
23
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
24 The trouble starts when you need to pass custom arguments through surefire down to the JVM that will run the tests. This is done through the `<argLine>` configuration parameter of maven-surefire-plugin:
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
25
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
26 :::xml
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
27 <plugin>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
28 <groupId>org.apache.maven.plugins</groupId>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
29 <artifactId>maven-surefire-plugin</artifactId>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
30 <configuration>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
31 <argLine>-Xmx128m</argLine>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
32 </configuration>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
33 </plugin>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
34
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
35 Looking back at the output of jacoco-maven-plugin above you'll notice that we're setting exactly the same `<argLine>` parameter as jacoco-maven-plugin does. If you don't take extra steps only one configuration can win - and that's the configuration you put into your pom. You have to run `mvn -X` to actually see this, though:
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
36
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
37 [DEBUG] Forking command line: /bin/sh -c cd /sonar-code-coverage && /opt/openjdk-bin-8.252_p09/jre/bin/java -Xmx128m -jar /sonar-code-coverage/target/surefire/surefirebooter9099169912642065145.jar /sonar-code-coverage/target/surefire 2020-05-12T07-39-04_225-jvmRun1 surefire3709014631518520641tmp surefire_01706858247537265191tmp
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
38
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
39 Note how the `-javaagent:...` argLine is not included in the command line arguments of the JVM that's forked to run the tests.
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
40
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
41 A way to fix this is through creative use of Maven build properties. Configure the jacoco-maven-plugin to set a different property than `argLine`:
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
42
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
43 :::xml
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
44 <plugin>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
45 <groupId>org.jacoco</groupId>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
46 <artifactId>jacoco-maven-plugin</artifactId>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
47 <configuration>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
48 <propertyName>surefire.argLine.sonar</propertyName>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
49 </configuration>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
50 </plugin>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
51
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
52 Next, configure maven-surefire-plugin to include that property into its `<argLine>`:
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
53
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
54 :::xml
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
55 <plugin>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
56 <groupId>org.apache.maven.plugins</groupId>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
57 <artifactId>maven-surefire-plugin</artifactId>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
58 <configuration>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
59 <argLine>-Xmx128m @{surefire.argLine.sonar}</argLine>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
60 </configuration>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
61 </plugin>
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
62
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
63 Note that you have to use [late evaluation](http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#argLine) of the property because it's not present when the Maven reactor starts but only after jacoco-maven-plugin has run.
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
64
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
65 Just one last hack is required to make the build work even if you don't run jacoco coverage as part of your build. Specify an empty property `surefire.argLine.sonar` in your project's properties so that it always exists and can be expanded.
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
66
6aa2fa328a94 New blog post on sonar and maven
Dirk Olmes <dirk.olmes@codedo.de>
parents:
diff changeset
67 For further reference I've put [a demo project up on github](https://github.com/dirk-olmes/sonar-code-coverage).