Visualize JavaScript code quality and code coverage with Sonar

It is hard to imagine a web project without JavaScript code today. JavaScript is an easy to learn and very performant script language. In the past we have used JavaScript mostly for eye-candy and form validation. Recently we have been asked more often to implement complex user interfaces with trees, sortable tables and things like that. So we decided to rely more on JavaScript to improve the feedback of the website to user actions.
The first question that came up was: How to develop test driven with JavaScript?
We decided to use Jasmine, a behaviour driven development framework which tests can be run headless in a Maven build for example.
The second question was: How to visualise code coverage and code quality?
The tool Sonar has been proven to be useful in our Java projects in the past. So the first I was searching for was the JavaScript Plugin for Sonar. It can be downloaded here and was luckily updated to version 1.0 recently with a bunch of new metrics like “avoid usage of == and !=”.
Unfortunately this plugin only supports JsTestDriver for code coverage and other test metrics. However, Jasmine support is on the Roadmap and I’m looking forward to see the next release of the JavaScript Plugin. But at the moment I had to integrate our Jasmine and jasmine-jquery tests with JsTestDriver, the JavaScript Plugin of Sonar and an automated maven build.

Used technologies

• Maven
• Jasmine
• jasmine-jstd-adapter
• JsTestDriver
• jstd-maven-plugin
• Sonar
• Sonar JavaScript Plugin

JsTestDriver

JsTestDriver is a test runner designed by Google and can be downloaded here. If you don’t know JSTestDriver yet, you maybe want to run over it’s documentation.
Some advantages of JsTestDriver:

• Eclipse and IntelliJ integration
• Maven Plugin
• parallel test executions across browsers (local or remote)
• code coverage

In order to run JsTestDriver you have to create a config file. By default you have to name it jsTestDriver.conf and you have to save it in src/test/resources. If you want to choose another filename or path remember to define these in the Maven plugin later.
In the config file you have to define following flags (Be sure that there are no whitespaces in front of the keywords):

server: http://localhost:9876
load:
  # jasmine dependency
  - lib/jasmine.js
  # to make our jasmine tests work within jstd
  # (must be included after jasmine.js)
  - lib/jasmineAdapter.js
  # models, views and other stuff
  - src/main/js/*.js
test:
  # load all test files
  - src/test/js/*Test.js
plugin:
  - name: "coverage"
    jar: "lib/coverage-1.3.4.b.jar"
    module: "com.google.jstestdriver.coverage.CoverageModule"

server

JsTestDriver starts it’s own server and generates a HTML page that can be captured by various browsers.

load

Define all needed JavaScript files like your models and views and so on. You can use a wildcard * to include all files within the specified directory. Note that it could be relevant to load your files in an correct order since a normal HTML page will be created and some dependencies have to be considered. The load sequence will be alphabetically if the wildcard is used.

test

simply attaches the test files to the created HTML page.

plugin

to be able to see the code coverage report in Sonar you have to download the coverage plugin and save it somewhere as you wish. Just remember to add the path to the plugin jar flag as shown in the listing above.

jstd-maven-plugin

Before we can see the Sonar report about the code quality and code coverage we have to configure maven to run jstd.
Unfortunately, the jstd-maven-plugin is not available at the Maven Central Repository. Therefore we have to add a new repository and pluginRepository to our pom.xml:

<repository>
    <id>jstd-maven-plugin google code repo</id>
    <url>http://jstd-maven-plugin.googlecode.com/svn/maven2</url>
</repository>
<pluginRepository>
    <id>jstd-maven-plugin google code repo</id>
    <url>http://jstd-maven-plugin.googlecode.com/svn/maven2</url>
</pluginRepository>

After that maven should be able to fetch the jstd-maven-plugin artifact:

<dependency>
    <groupId>com.googlecode.jstd-maven-plugin</groupId>
    <artifactId>jstd-maven-plugin</artifactId>
    <version>1.3.2.5</version>
    <scope>test</scope>
</dependency>

To run our tests with a maven build we need the jstd-maven-plugin as a Maven plugin:

<plugin>
    <groupId>com.googlecode.jstd-maven-plugin</groupId>
    <artifactId>jstd-maven-plugin</artifactId>
    <version>1.3.2.5</version>
    <configuration>
        <browser>firefox</browser>
        <port>9876</port>
        <testOutput>target/jstestdriver</testOutput>
    </configuration>
    <executions>
        <execution>
            <id>run-tests</id>
            <goals>
                <goal>test</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Three configuration flags are mandatory. More command line flags can be found in the documentation of jstd.

browser

a comma separated list of browsers (more exactly the path to the specific browser) that should be used for the tests

port

the port that is set in jsTestDriver.conf

testOutput

This specifies the directory where the code coverage reports (needed for Sonar) will be saved. The default directory for sonar is target/jstestdriver, so remember to configure sonar accordingly, if you choose another directory.

Set the sourceDirectory in the pom.xml

In order for Sonar to be able to analyze the JavaScript code and to visualize the reports, we must add the path to the source code which is src/main/js in our case.

<build>
    <sourceDirectory>src/main/js</sourceDirectory>
    <!-- ... -->
</build>

Run the tests and the analysis

Everything should be configured correctly now. So just start the maven build:

mvn jstd:test

JsTestDriver opens the defined browsers, runs all tests and generates the code coverage report. After that we have to start the sonar build:

mvn sonar:sonar -Dsonar.language=js -Dsonar.branch=js

To tell sonar to analyze a JavaScript project the sonar.language property is essential. If the same project should be analyzed as a Java project you may want to add a branch with the property sonar.branch. Otherwise the previous values will be overridden with this JavaScript analysis.

Problems

An annoying problem is running the tests with real browsers like Firefox and Chrome. The maven build automatically starts the browser and also closes it after the tests are finished. But Firefox is not correctly closed by jstd somehow… so the next test run fails because Firefox opens a dialog which must be closed manually. The maven build is deadlocked and you have to abort and rerun it…
So maybe a running Firefox process would be a workaround, I thought. Well, it kinda worked but the opened tab was not closed anymore (tested on Linux and Windows). Each new test run opened a new tab and after a handful testruns the tests failed because of some strange error. Closing tabs manually solved this, however. Same problem occurred with Chrome (Version 22.0.1201.0 dev).
On one hand it is really nice to run the tests in all desired browsers, on the other hand closing tabs/browsers manually makes it impossible to automate this process. So I’m really looking forward to Jasmine support of the sonar JavaScript Plugin to run headless tests as a maven build, just like jasmine-maven-plugin. A quick google search links to this project: https://github.com/jwark/jstd-standalone-headless Maybe this could be a solution… Any information is welcome, so if you have a working setup, please let me know.
Another problem surely is the mandatory specification of the sourceDirectory to be able to see the metrics in Sonar. Usually you will have a Java project with some JavaScript code. Therefore you certainly can’t pinpoint to src/main/js as source directory of the project, for example. Further information is appreciated, again 🙂

Todo

automate the analysis within a Jenkins build process
— maybe jstd tests can be run headless?
— maybe maven profiles could be used to prevent the sourceDirectory declaration?
require.js integration in jstd-unit-tests
— RequireJS is a JavaScript file and module loader and this should be a good starting point.