JarCasting
JarCasting JarCasting

Serenity JUnit5

Writing and running Serenity BDD tests with JUnit5 (Jupiter test engine)

License

License
Categories

Categories

JUnit Unit Testing
GroupId

GroupId

io.github.fabianlinz
ArtifactId

ArtifactId

serenity-junit5
Last Version

Last Version

1.6.0
Release Date

Release Date
Type

Type

pom.sha512
Description

Description

Serenity JUnit5
Writing and running Serenity BDD tests with JUnit5 (Jupiter test engine)
Project URL

Project URL

https://github.com/fabianlinz/serenity-junit5
Source Code Management

Source Code Management

https://github.com/fabianlinz/serenity-junit5

Download serenity-junit5

Filename Size
serenity-junit5-1.6.0.pom
serenity-junit5-1.6.0-sources.jar 16 KB
serenity-junit5-1.6.0-javadoc.jar 86 KB
Browse

Dependencies

compile (2)

Group / Artifact Type Version
net.serenity-bdd : serenity-core jar 2.3.9
org.junit.jupiter : junit-jupiter-api jar 5.7.0

Project Modules

There are no modules declared in this project.

master build status maven-central

This is a early version of JUnit5 support for Serenity BDD. Feedback and help are highly appreciated.

Writing and running Serenity BDD tests with JUnit5 (Jupiter test engine)

The Serenity BDD features described for JUnit4 (http://thucydides.info/docs/serenity-staging/#_serenity_with_junit) should also work for JUnit5 (unless mentioned in the known limitations section below).

Aspects specific to JUnit4 (e.g. packages for the @Test annotation or the name of lifecycle annotations) are different of cause (see also https://junit.org/junit5/docs/current/user-guide/#migrating-from-junit4-tips).

To run a JUnit5 test with Serenity BDD, simply add the annotation @net.serenitybdd.junit5.SerenityTest (instead of @org.junit.runner.RunWith(net.serenitybdd.junit.runners.SerenityRunner.class) for JUnit4)

Basic example (analog to http://thucydides.info/docs/serenity-staging/#_basic_junit_integration):

@SerenityTest                                                       
public class WhenCalculatingFrequentFlyerPoints {

    @Steps                                                                              
    TravellerSteps travellerSteps;

    @Test
    void shouldCalculatePointsBasedOnDistance() {
        // GIVEN
        travellerSteps.a_traveller_has_a_frequent_flyer_account_with_balance(10000);    

        // WHEN
        travellerSteps.the_traveller_flies(1000);                                       

        // THEN
        travellerSteps.traveller_should_have_a_balance_of(10100);                       

    }
}

To get started just include a dependency to io.github.fabianlinz:serenity-junit5.

Adding a direct dependency to serenity-core allows to easily use newer versions of Serenity. The transitive dependency to JUnit4 should be excluded though:

    testImplementation ('io.github.fabianlinz:serenity-junit5:VERSION')
    testImplementation ('net.serenity-bdd:serenity-core:VERSION') { // optional: in case a newer version than the transitively included should be used
        exclude group: 'junit', module: 'junit'
    }

Why JUnit5 support for Serenity BDD?

  • Serenity BDD supports different options to define and run tests e.g. Cucumber, JBehave, JUnit.
  • While Serenity BDD with JUnit4 style tests can be executed on the JUnit5 platform using the JUnit 5 vintage test engine (see also https://junit.org/junit5/docs/current/user-guide/#migrating-from-junit4-running) the syntax is different in several aspects. For teams using JUnit to declare the Serenity scenarios, the difference is sometimes tedious. For example
    • JUnit5 does not require public methods and classes
    • different annotations are used for the lifecycle hooks
    • different extension mechanism: JUnit4 Rule vs. JUnit5 Extension

Notes on supported features

Junit5 @Disabled vs. Serenity @Pending

  • Junit5 @Disabled annotation can be used on test and step methods

  • The following table outlines the difference of using JUnit5 @Disabled and Serenity @Pending on a step method

    Consequence of annotated step method for... JUnit5 @Disabled (same as @Ignore in JUnit4) Serenity @Pending
    test outcome ignored pending
    annotated step method skipped (= not executed) skipped (= not executed)
    annotated step method in report ignored pending
    step methods after the annotated step executed skipped (= not executed)
    step methods after the annotated step in report depending on execution e.g. success ignored

Tagging

  • Serenity and Junit5 provide annotations to tag tests and test classes.
    • The tags are shown in the Serenity report and can be used to filter the tests to be executed.
      • In contrast to Serenity @WithTag the JUnit5 @Test annotation is repeatable. So multiple tags can be defined on a method or class without having to use a container annotation (Serenity: @WithTags, Junit5 (optional): @Tags)
      • Junit5 @Test can also be used on meta-annotations.
  • In Serenity tags have a type and a name. For the Serenity report a Junit5 @Tag value is parsed like a Serenity @WithTags value. So it is also possible to define a type using Junit5 @Tag.

    • Examples

      Serenity Junit5 Result in Serenity report: type Result in Serenity report: name
      @WithTag("tagName")

      @WithTag(type="tag" name="tagName")

      @WithTagValuesOf({"tagName"})      		 @Tag("tagName") tag tagName
      @WithTag(name="tagName")

      @WithTag("feature:tagName")

      @WithTagValuesOf({"feature:tagName"})      		 @Tag("feature:tagName") feature tagName
      @WithTag("tagType=tagName")

      @WithTag("tagType:tagName")

      @WithTag(type="tagType" name="tagName")

      @WithTagValuesOf({"tagType=tagName"})

      @WithTagValuesOf({"tagType:tagName"})      		 @Tag("tagType=tagName")

      @Tag("tagType:tagName")      		 tagType tagName
  • see also

Serenity @Manual

  • Tests (not steps) can be annotated with @Manual.
    • @Manual on a test method has no effect if the test method is also annotated with @Disabled or @Pending (consistent with JUnit4)
    • In contrast to Junit4 a test method annotated with @Manual will actually be executed. This allows to further specify the example using @Step methods and show them the report. While this is inconsistent with Junit4 support in Serenity, it is consistent with the Cucumber support (see also https://serenity-bdd.github.io/theserenitybook/latest/manual-tests.html).

Other

Known limitations/currently not supported features:

Serenity BDD features

  • @WithTag and @WithTagValuesOf (http://thucydides.info/docs/serenity-staging/#_filtering_test_executing_with_tags)
    • Filtering of tests not yet possible => works with JUnit5 @Tag though (#22)
  • SerenityParameterizedRunner (including Serenity @Concurrent)
  • Retrying failed tests
  • Difference for WebDriver configuration on method level (compared to JUnit4)
    • If the test method is annotated with @WithDriver the specified driver is also used for an @Managed field of type WebDriver without an explicit driver type. In JUnit4 the @Managed field would be the general default web driver. As this seems arbitrary anyway this difference seems to be acceptable in order to not further complicate the implementation.
  • Serenity annotation @UsePersistantStepLibraries
    • JUnit4: SerenityRunner#runChild >> SerenityRunner#resetStepLibrariesIfRequired() + net.serenitybdd.junit5.extension.page.TestConfiguration#shouldResetStepLibraries (be careful with dependency to page stuff)
  • Extension point analog to JUnit4 SerenityRunner#additionalBrowserCleanup

JUnit5 features

  • @Nested
  • @DisplayName
    • on test method level: works = does control the scenario name
    • on test class level: is NOT considered for the story name
      • issue: there is no API to manipulate net.thucydides.core.model.Story.from (which is called via BaseStepListener.testSuiteStarted(java.lang.Class<?>)) (implicit assumption: JUnit4 is used). Would be great if there were an SPI to hook in other strategies for determining the name. Setting the Story directly has the downside that the BaseStepListener is unaware of the test class.

Not yet verified/analysed features

Serenity BDD

JUnit5

General notes:

  • Should there be an alternative for identifying WebTests?
    • currently a test is a web test if the test has at least one field of type WebDriver that is annotated with @Managed
    • downside of this approach: test has to declare a field that it does not need
    • options
      • A) Always activate: downside is that it adds some initialization/performance overhead => if is the only downside this might not be a real issue
      • B) Allow alternative strategies to activate web test support (e.g. annotation or naming pattern)
  • Is the WebDriver configuration on test class and method level still relevant?
    • configuration options
      • serenity.properties
      • @Manager can be used to set dirver name and options on test class level
      • @WithDriver + @DriverOptions can be used on method level
    • The option to configure on method level
    • Proposal: deprecate @WithDriver + @DriverOptions
  • Is it a common use case to have multiple web driver fields in a test class?
    • What would I expect to be the effective web driver in Steps?
    • At the moment this seems to be arbitrary (depending on which field Java reflection provides as the "first" field)
    • See also comments in net.serenitybdd.junit5.extension.page.SerenityPageExtension
  • Using SerenityRunner#methodInvoker would allow to reduce the code but it would not allow to control the webdriver on test method level (via @WithDriver).
    • Additional complexity accepted in favour of higher consistency with JUnit4 support.
  • SerenityStepExtension#interceptTestMethod => Different to JUnit4 in case step assertion failure or exception is not thrown in test method itself but e.g. a @BeforeEach/@AfterEach
  • Various experimental features of JUnit5 were used (https://junit.org/junit5/docs/current/user-guide/#api-evolution-experimental-apis)
    • org.junit.jupiter.api.extension.InvocationInterceptor
    • org.junit.jupiter.api.extension.LifecycleMethodExecutionExceptionHandler
    • org.junit.jupiter.api.extension.TestWatcher
  • Not integrated because currently only used for tests and not necessary for JUnit5
    • net.serenitybdd.junit.runners.SerenityRunner#clearMetadataIfRequired (net.serenitybdd.junit5.extension.page.TestConfiguration#shouldClearMetadata) net.serenitybdd.core.Serenity#getCurrentSession
  • serenity-core and serenity-model versions before 2.0.84 have a runtime dependencies to JUnit4. This was removed by #1884

Versions

Version
1.6.0
1.5.0
1.4.0
1.3.0
1.2.1
1.2.0