Implementing acceptance tests with jbehave

Producing high quality software in an agile process means that everybody involved in the delivery team (or in other words: the team as a whole) do their best to ensure that each products increment delivered to the customer meets the business values of each story that has been implemented.
One strategy to achieve this, is to define executable specifications each reflecting a stories acceptance criteria. Doing so allows you to document the story requirements and drive the development of the product – this is what behaviour driven development (BDD) is about. Not only does an executable specification provide a definition-of-done for a given story; in addition, it will serve as a regression test, ensuring the acceptance criteria are fulfilled while the product evolves in following sprints.
An important difference between automated acceptance tests and unit- (or component-) tests is that acceptance tests are business-facing, i.e. their responsibility is to make sure that each story implementation delivers a certain business value to the customer. Ideally, they are written by the customer (with the help of business analysts and/or testers) and are implemented by the testers, for example by pairing with programmers. There are various tools available supporting the creation of automated acceptance tests, most of them providing a DSL (internal or external).
jbehave is one of these tools, allowing to define a stories acceptance criteria in the commonly used given-when-then form. The following snippets show how a jbehave acceptance test might look like, how it is implemented and integrated into a project. To keep things simple the acceptance test verifies the behaviour of a certain service, directly using the service layers API. In a real project – where the behaviour of an application running in a test environment should be verified – the acceptance test probably would not be written against the serive API. Instead, an application driver would be injected into the acceptance test, in order to initialize application state and then make requests against the applications REST-API or the GUI (using a window driver component).
So, let’s start by adding the required dependencies and plugins into the POM (at this point, it should be obvious that my demo project is Maven-based):

...
<modelVersion>4.0.0</modelVersion>
<groupId>dyn-ip</groupId>
<artifactId>dyn-ip</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>war</packaging>
<name>DynIP</name>
...
<dependency>
  <groupId>junit</groupId>
  <artifactId>junit</artifactId>
  <scope>test</scope>
</dependency>
<!-- enables jbehave acceptance tests -->
<dependency>
  <groupId>org.jbehave</groupId>
  <artifactId>jbehave-core</artifactId>
  <version>3.6.8</version>
  <scope>test</scope>
</dependency>
<!-- adds dependency injection support
     using the Weld CDI container -->
<dependency>
  <groupId>org.jbehave</groupId>
  <artifactId>jbehave-weld</artifactId>
  <version>3.6.8</version>
  <scope>test</scope>
</dependency>
...

The jbehave-weld dependency allows to inject dependencies into our Steps classes (Steps and Stories are shown below). Finally, the jbehave-maven-plugin is included into the POM and configured to match and execute Stories classes at the integration-test phase using the plugins run-stories-with-annotated-embedder goal:

...
<plugin>
  <groupId>org.jbehave</groupId>
  <artifactId>jbehave-maven-plugin</artifactId>
  <version>3.6.8</version>
  <executions>
    <execution>
      <phase>integration-test</phase>
      <configuration>
        <includes>
          <include>**/*Stories.java</include>
        </includes>
        <scope>test</scope>
        <testSourceDirectory>src/integrationtest/java</testSourceDirectory>
      </configuration>
      <goals>
        <goal>run-stories-with-annotated-embedder</goal>
      </goals>
    </execution>
  </executions>
  <dependencies>
    <dependency>
      <groupId>org.jbehave</groupId>
      <artifactId>jbehave-weld</artifactId>
      <version>3.6.8</version>
    </dependency>
  </dependencies>
</plugin>
...

Note that in my demo project, all jbehave related classes and resources are located in the src/integrationtest/java/ and src/integrationtest/resources/ directories respectively. In order for this to work, you also have to include the build-helper-maven-plugin into the POM. In a real project it would probably be better to have a separate project for your acceptance tests. Now, we will define some acceptance criteria stored in the file src/integrationtest/resources/stories/dynip.stories:

Scenario: get an ip address (host unspecified)
Given an initialized network service
When no host is specified
Then the result should be any valid ip address
Scenario: get an ip address for host
Given an initialized network service
When the host is localhost
Then the address should be 127.0.0.1

As you can see, the acceptance test is about verifying the behaviour of some network service. Here, the acceptance criteria for two scenarios are written down in a given-when-then form. Next, we create an implementation of the scenario steps in a Steps class (which is just a POJO):

@WeldStep
@Singleton
public class DynIpSteps {
private static ValidatorFactory validatorFactory;
  static {
    validatorFactory = buildDefaultValidatorFactory();
  }
  @Inject
  private NetworkService networkService;
  private String host;
  @Given("an initialized network service")
  public void anInitializedNetworkService() {
  }
  @When("no host is specified")
  public void whenNoHostIsSpecified() {
    this.host = null;
  }
  @Then("the result should be any valid ip address")
  public void thenTheResultShouldBeAnyValidIpAddress() {
    Validator validator = validatorFactory.getValidator();
    Set<ConstraintViolation<Ipv4Address>> violations =
      validator.validate(networkService.getIpv4Address());
    assertThat(violations.size(), is(0));
  }
  @When("the host is $host")
  public void whenTheHostIs$Host(String host) {
    this.host = host;
  }
  @Then("the address should be $address")
  public void thenTheAddressShouldBe$Address(String address) {
    Ipv4Address ip = networkService.getIpv4AddressForHost(host);
    assertThat(ip.getTextualRepresentation(), is(address));
  }
}

While the class level annotations are related to dependency injection, the method level annotations represent the various steps defined in the stories scenarios.
Along with the Steps class we also implement a Stories class which knows about both, the story files and the Steps classes. In order to reduce boilerplate code, an abstract base class is extended by each Stories class:

@UsingSteps(instances = {DynIpSteps.class})
public class DynIpStories extends StoriesBase {
  public DynIpStories() {
    super("**/*.stories");
  }
}
@RunWith(WeldAnnotatedEmbedderRunner.class)
@Configure
@UsingWeld
@UsingEmbedder(embedder = Embedder.class,
  generateViewAfterStories = true,
  ignoreFailureInStories = true,
  ignoreFailureInView = true)
public abstract class StoriesBase extends InjectableEmbedder {
  private final String storiesPath;
  private final URL codeLocation;
  protected StoriesBase(String storiesPath) {
    this.storiesPath = storiesPath;
    this.codeLocation = codeLocationFromClass(this.getClass());
  }
  @Test
  @Override
  public void run() throws Throwable {
    StoryFinder storyFinder = new StoryFinder();
    injectedEmbedder().runStoriesAsPaths(
      storyFinder.findPaths(codeLocation, storiesPath, ""));
  }
}

Since we are using jbehave annotations it’s important to set the goal element to run-stories-with-annotated-embedder in the jbehave-maven-plugins configuration (as shown above). Now, there are only a few configuration related classes to be implemented: one class that produces the configuration information and two other classes that relate to story loading and report building:

@ApplicationScoped
public class ConfigurationProducer {
@Produces @WeldConfiguration
public Configuration getConfiguration() {
  return new MostUsefulConfiguration()
    .useStoryLoader(new StoryLoader())
    .useStoryReporterBuilder(new ReportBuilder());
  }
}
public class StoryLoader extends LoadFromClasspath {
  public StoryLoader() {
    super(StoryLoader.class.getClassLoader());
  }
}
public class ReportBuilder extends StoryReporterBuilder {
  public ReportBuilder() {
    withFormats(CONSOLE, TXT, HTML).withDefaultFormats();
  }
}

The Steps class must be annotated with annotation javax.inject.Singleton (this is because in my demo project I’m using the Weld CDI container which does not default to create singleton scoped beans; having no singleton prevents us from holding any state information across multiple calls of the @Given, @When and @Then annotated methods). For the dependency injection mechanism to work, there must also be (an empty) beans.xml file in both, directory src/integrationtest/resources/META-INF/ and directory src/main/resources/META-INF/