Java Code Coverage Tools

Java Code Coverage Tools are distinguished in two main categories: first tools that add statements to the source code and require to recompile the source code. And second, tools which instrument the byte code either before or while running it. The goal is to find out which parts of code are tested by registering the lines of code executed when doing the test.

JCov

JCov
Developer(s) Leonid Arbouzov, Alexander Petrov, Vladimir Generalov, Serguei Chukhontsev, Oleg Uliankin, Gregory Steuck, Pavel Ozhdikhin, Konstantin Bobrovsky, Robert Field, Alexander Kuzmin, Leonid Mesnik, Sergey Borodin, Andrey Titov, Dmitry Fazunenko, Alexey Fedorchenko
Stable release 3.0 / September 1, 2014 (2014-09-01)
Operating system Cross-platform
Type Code coverage
License GNU Public License, version 2, with the Classpath Exception
Website JCov

JCov is the tool which has been developed and used with Sun JDK (and later Oracle JDK) from the very beginning of Java: from the version 1.1. JCov is capable of measuring and reporting Java code coverage. JCov is distributed under the terms of the GNU Public License, version 2, with the Classpath Exception. JCov has been open-sourced as a part of OpenJDK codetools project in 2014. JCov is the only code coverage tool working with a JDK release in development (JDK9 at the time of writing).

Features

JCov is capable of reporting the following types of code coverage:

JCov implements two different ways to save the collected data:

JCov works by instrumenting Java bytecode using two different approaches:

JCov has a few more distinctive features which include, but are not limited to:

Tools using JCov

JaCoCo

JaCoCo
Developer(s) Marc Hoffmann, Brock Janiczak, Evgeny Mandrikov, Mirko Friedenhagen
Stable release 0.7.6 / February 18, 2016 (2016-02-18)
Operating system Cross-platform
Type Code coverage
License EPL
Website EclEmma JaCoCo

JaCoCo is an open source toolkit for measuring and reporting Java code coverage. JaCoCo is distributed under the terms of the Eclipse Public License. It was developed as a replacement for EMMA[1] under the umbrella of the EclEmma eclipse project.

Features

JaCoCo offers line and branch coverage. In contrast to Clover, which requires instrumenting the source code, and Cobertura, which instruments the bytecode offline, JaCoCo instruments the bytecode while running the code. To do this it runs as a Java agent,[2] and can be configured to store the collected data in a file, or send it via TCP. Files from multiple runs or code parts can be merged easily.[3] Unlike Cobertura and Emma it fully supports Java 7 and Java 8.[4]

Tools using or including JaCoCo

Clover

Clover
Developer(s) Atlassian
Stable release 4.0.4 / April 20, 2015 (2015-04-20)
Operating system Cross-platform
Type Code coverage
License Proprietary
Website www.atlassian.com

Clover is a Java Code Coverage Analysis application bought and further developed by Atlassian. It is a commercial product freely available to open source projects and non-profit institutions.

Clover is using a source code instrumentation technique (as opposed to Cobertura and JaCoCo which are using byte code instrumentation), which has its advantages (like an ability to collect code metrics) and disadvantages (re-compilation of sources is necessary). [9] Some features include historical reporting, huge control over the coverage gathering process, command line toolset and API for legacy integration and more.

Clover also allows testing time to be reduced by only running the tests that cover the application code that was modified since the previous build. This is called Test Optimization and can lead to huge drops in the amount of time spent waiting for automated tests to complete.

Clover comes with a number of integrations both developed by Atlassian (Ant, Maven, Grails, Eclipse, IDEA, Bamboo) and by open source community (Gradle, Griffon, Jenkins, Hudson, Sonar).


In order to run Clover from Apache Ant you can add the following to Ant build.xml file: [10]

<taskdef resource="cloverlib.xml" classpath="${clover.jar}"/>
<clover-env/>

and next run:

ant clover.all

In order to run Clover from Apache Maven you can add the following to the ~/.m2/settings.xml: [11]

<settings ...>
  <pluginGroups>
    <pluginGroup>com.atlassian.maven.plugins</pluginGroup>
  </pluginGroups>
  ...
</settings>

and next type:

mvn clean clover2:setup test clover2:aggregate clover2:clover

Cobertura

Cobertura
Developer(s) Steven Christou
Stable release 2.1.1 / February 26, 2015 (2015-02-26)
Operating system Cross-platform
Type Code coverage
License GPL 2.0
Website cobertura.github.io/cobertura/

Cobertura is an open source tool for measuring code coverage. It does so by instrumenting the byte code.

EMMA

EMMA
Developer(s) Vlad Roubtsov
Stable release 2.1 / May 13, 2005 (2005-05-13)
Operating system Cross-platform
Type Code coverage
License Common Public License 1.0
Website emma.sourceforge.net

EMMA is an open source toolkit for measuring and reporting Java code coverage. EMMA is distributed under the terms of Common Public License v1.0.

EMMA is not currently under active development; the last stable release took place in mid-2005. As replacement, JaCoCo was developed.[12] EMMA works by wrapping each line of code and each condition with a flag, which is set when that line is executed.[13]

Features

EMMA-based tools

Serenity

Serenity
Developer(s) Michael Couck
Stable release 1.0 / December 8, 2013 (2013-12-08)
Operating system Cross-platform
Type Code coverage
License Apache Software License, Version 2.0
Website wiki.jenkins-ci.org/display/JENKINS/Serenity+Plugin

Serenity is an open source toolkit for measuring and reporting Java code coverage. As well as coverage, major code metrics are measured:- cyclometric complexity, stability, abstractness and distance from main. The report data is persisted to an object database, and made available via Jenkins/Hudson. The interface replicates the Eclipse IDE interface visually.

Serenity dynamically enhances the byte code, making a post-compile step unnecessary. Ant and Maven projects are supported. Configuration is done in xml, an Ant example would be:

    <!-- Serenity system properties. -->
    <sysproperty key="included.packages" value="your.package.name.here" />
    <sysproperty key="included.adapters" value="coverage,complexity,dependency" />
    <!-- Serenity JVM command line. -->
    <jvmarg line="-javaagent:serenity/serenity.jar" />

And a Maven configuration example would be:

    <properties>
	<included.packages>-Dincluded.packages=your.package.name.here</included.packages>
	<included.adapters>-Dincluded.adapters=coverage,complexity,dependency</included.adapters>
    </properties>
    <argLine>-javaagent:serenity/serenity.jar -Xms512m -Xmx1024m ${included.packages} ${included.adapters}</argLine>

For a full example of a configuration please refer to the Jenkins wiki at https://wiki.jenkins-ci.org/display/JENKINS/Serenity+Plugin.

Jenkins slaves as well as Maven multi module projects are supported.

References

  1. ↑ JaCoCo Mission
  2. ↑ Patroklos Papapetrou (19 December 2012). "Code Coverage Tools (JaCoCo, Cobertura, Emma) Comparison in Sonar". Only Software matters. Retrieved 3 March 2013.
  3. ↑ Measure Coverage by Integration Tests with Sonar – Updated
  4. 1 2 "Code Coverage". IntelliJ IDEA 12.0 Web Help. JetBrains. Retrieved 3 March 2013.
  5. ↑ EclEmma, Eclipse code coverage plugin
  6. ↑ Jenkins JaCoCo Plugin
  7. ↑ NetBeans JaCoCo support
  8. ↑ Gradle JaCoCo Plugin
  9. ↑ Why does Clover use source code instrumentation?
  10. ↑ Clover-for-Ant two line integration
  11. ↑ Clover-for-Maven quick start guide
  12. ↑ EMMA code coverage files on SourceForge.net
  13. ↑ Expert Spring MVC and Web Flow; By Seth Ladd, Darren Davison, Steven Devijver, Colin Yates, p. 289

External links

This article is issued from Wikipedia - version of the Wednesday, April 27, 2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.