JarCastingCucumber/BDD testing for REST APIs
This project provides a generic Cucumber vocabulary for testing APIs with the Ready! API TestServer, with special support for Swagger to remove some of the technicalities required to define scenarios.
A quick example for the Petstore API at http://petstore.swagger.io, testing of the /pet/findByTags resource could be defined withe following Scenario:
Scenario: Find pet by tags
Given the API running at http://petstore.swagger.io/v2
When a GET request to /pet/findByTags is made
And the tags parameter is test
And the request expects json
Then a 200 response is returned within 50ms
And the response type is json
Using the integrated Swagger support this can be shortened to
Scenario: Find pet by tags
Given the Swagger definition at http://petstore.swagger.io/v2/swagger.json
# deducts path and method from Swagger definition by operationId
When a request to findPetsByTags is made
# deducts type of "tags" parameter (query/path/parameter/body) from Swagger definition
And tags is test
And the request expects json
Then a 200 response is returned within 500ms
And the response type is json
Not a huge difference - but as you can see by the comments the Swagger support removes some of the technicalities; read more about Swagger specific steps below!
Check out the samples submodule for more examples.
Usage with maven/junit
If you want to run scenarios as part of a maven build you need to add the following dependency to your pom:
<dependency>
<groupId>com.smartbear.readyapi.testserver.cucumber</groupId>
<artifactId>testserver-cucumber-stepdefs</artifactId>
<version>1.0.0</version>
<scope>test</scope>
</dependency>
Then create a JUnit runner class that uses Cucumber and point it to your feature files:
@RunWith(Cucumber.class)
@CucumberOptions(
plugin = {"pretty", "html:target/cucumber"},
features = {"src/test/resources/cucumber"})
public class CucumberTest {
}
(see the included samples module for a working project)
Running from the command-line
If you don't want to run your tests as part of a java/maven/etc-build or simply want to run them from the command-line you can use the testserver-cucumber-runner jar file which includes all required libraries including the Cucumber runtime. Run tests with:
java -jar testserver-cucumber-runner-1.0.0.jar <path to feature-files>
Internally this will call the regular cucumber.api.cli.Main class with an added -g argument to the included glue-code, all other options are passed as usual, see https://cucumber.io/docs/reference/jvm#cli-runner
(you will need java8 installed on your path)
Running with Docker
If the above two options are too much of a java-hassle for you, then you can use the corresponding docker image available at https://hub.docker.com/r/smartbear/cucumber4apis instead - it packages the above runner and makes it super-easy to run feature files for your APIs, for example:
docker run -v /Users/Ole/cucumber:/features smartbear/cucumber4apis -p pretty /features
Here I mounted my local folder containing feature files into a volume named "/features" in the container - and then specify that that volume as the source for feature files for the Cucumber Runner (together with the -p pretty argument).
Recipe logging
If you add a -Dtestserver.cucumber.logfolder=... system property to your command line invocation the runner will write generated json recipe files to the specified folder before sending them to the TestServer, for example allowing you to import them into Ready API for load-testing/monitoring/etc.
Configuring Ready! API TestServer access
The included Cucumber StepDefs build and execute test recipes agains the Ready! API TestServer using the testserver-java-client, by default they will submit recipes to the publicly available TestServer at http://testserver.readyapi.io. If you want to run against your own TestServer instance to be able to access internal APIs or not run into throttling issues you need to download and install the TestServer from https://smartbear.com/product/ready-api/testserver/overview/ and configure access to it by specifying the corresponding system properties when running your tests:
- testserver.endpoint=...url to your testserver installation...
- testserver.user=...the configured user to use...
- testserver.password=...the configured password for that user...
(these are picked up by CucumberRecipeExecutor during execution)
Building
Clone this project and and run
mvn clean install
To build and install the artefacts in your local maven repository - the packaged jar is created in the root target folder.
API Testing Vocabulary
The included glue-code for API testing adds the following vocabulary:
Given statements
-
"the Swagger definition at <swagger endpoint>"
- The specified endpoint must reference a valid Swagger 2.0 definition
- Example: "the Swagger definition at http://petstore.swagger.io/v2/swagger.json"
-
"the API running at <API endpoint>"
- Example: "the API running at http://petstore.swagger.io/v2"
-
"the oAuth2 token <token>"
- Example: "the oAuth2 token 18273827aefef123"
- Adds an OAuth 2.0 Bearer token to requests
When/And statements
-
"a <HTTP Method> request to <path> is made"
- Example: "a GET request to /test/search is made"
-
"a request to <Swagger OperationID> is made"
- will fail if no Swagger definition has been Given
- Example: "a request to findPetById is made"
-
"the request body is" <text block>
- Example: "the request body is
""" { "id" : "123" } """ ```" -
"the <parameter name> parameter is <parameter value>"
- adds the specified parameter as a query parameter
- Example: "the query parameter is miles davis"
-
"the <http header> is <header value>
- Example: "the Encoding header is UTF-8"
-
"the type is <content-type>
- single word types will be expanded to "application/<content-type>"
- Example: "the type is json"
-
"<parameter name> is <parameter value>"
- if a valid OperationId has been given the type of parameter will be deduced from its list of parameters
- if no OperationId has been given this will be added to a map of values that will be sent as the request body
- Example: "name is John"
- will work for both inline or multi-line values
-
"<the request expects <content-type>"
- adds an Accept header
- Example "the request expects yaml"
Then/And statements:
-
"a <HTTP Status code> response is returned"
- Example: "a 200 response is returned"
-
"a <HTTP Status code> response is returned within <number>ms"
- Example: "a 404 response is returned within 10ms"
-
"the response is <a valid Swagger Response description for the specified operationId>"
- Requires that a valid OperationId has been Given
- Example: "the response is a list of people"
-
"the response body contains" <text block>
- Example: "the response body contains
""" "id" : "123" """ ```" -
"the response body matches" <regex text block>
- Example: "the response body matches
""" .*testing.* """ ```" -
"the response type is <content-type>"
- Example: "the response type is application/yaml"
-
"the response contains a <http-header name> header"
- Example: "the response contains a Cache-Control header"
-
"the response <http header name> is <http header value>"
- Example: "the response Cache-Control header is None"
-
"the response body contains <text token>"
- Example: "the response body contains Testing text"
Complete Example:
Below is the swaggerhub.feature in the samples submodule.
Feature: SwaggerHub REST API
Background:
Given the Swagger definition at https://api.swaggerhub.com/apis/swagger-hub/registry-api/1.0.10
Scenario: Default API Listing
When a request to searchApis is made
Then the response is a list of APIs in APIs.json format
Scenario: Owner API Listing
When a request to getOwnerApis is made
And owner is swagger-hub
Then the response is a list of APIs in APIs.json format
Scenario: API Version Listing
When a request to getApiVersions is made
And owner is swagger-hub
And api is registry-api
Then the response is a list of API versions in APIs.json format
And the response body contains
"""
"url":"/apis/swagger-hub/registry-api"
"""
Scenario Outline: API Retrieval
When a request to getDefinition is made
And owner is <owner>
And api is <api>
And version is <version>
Then a 200 response is returned within 500ms
And the response type is json
And the response body contains
"""
"description":"<description>"
"""
Examples:
| owner | api | version | description |
| swagger-hub | registry-api | 1.0.10 | The registry API for SwaggerHub |
| fehguy | sonos-api | 1.0.0 | A REST API for the Sonos platform |
Extending the vocabulary
You can extend the supported Gherkin vocabulary by providing custom StepDefs that tie into the underlying TestServer recipe generation. Do this as follows (a complete example is shown below):
- Create a Custom StepDefs class which you annotate with @ScenarioScoped
- Create a Constructor into which you inject an instance of CucumberRecipeBuilder
- Implement your Given/When/Then/And methods to build TestSteps and add them to the builder provided to the constructor
Internally the actual recipe gets created and sent to the TestServer first in a Cucumber @After handler
If you want to delegate some of your custom vocabulary to the existing RestStepDefs you can inject them into your custom StepDefs constructor also and then use it as needed.
Javadocs for related classes are available at http://readyapi.github.io/testserver-cucumber/apidocs/
The below class shows all the above concepts:
package com.smartbear.samples.cucumber.extension;
import com.smartbear.readyapi.testserver.cucumber.CucumberRecipeBuilder;
import com.smartbear.readyapi.testserver.cucumber.RestStepDefs;
import cucumber.api.java.en.Given;
import cucumber.runtime.java.guice.ScenarioScoped;
import javax.inject.Inject;
@ScenarioScoped
public class CustomStepDefs {
private final CucumberRecipeBuilder recipeBuilder;
private final RestStepDefs restStepDefs;
@Inject
public CustomStepDefs(CucumberRecipeBuilder recipeBuilder, RestStepDefs restStepDefs ){
this.recipeBuilder = recipeBuilder;
this.restStepDefs = restStepDefs;
}
/**
* Provide an alternative vocabulary for specifying an API endpoint
*/
@Given("^an endpoint of (.*)$")
public void anEndpointOf( String endpoint ) throws Throwable {
restStepDefs.setEndpoint( endpoint );
}
}
To get this used during execution you will need to
- Compile the above into a jar file
- Include the jar file in the classpath for the TestServer Cucumber runner
- Add the containing package(s) of your StepDefs with the -g argument
For example (line-breaks and comments added for readability):
java -cp modules/samples/target/testserver-cucumber-samples-1.0.1-SNAPSHOT.jar: // the extension jar
modules/runner/target/testserver-cucumber-runner-1.0.1-SNAPSHOT.jar // the runner jar
com.smartbear.readyapi.testserver.cucumber.CucumberRunner // the runner class
-g com.smartbear.samples.cucumber.extension // the extension package
modules/samples/src/test/resources/cucumber // the features folder
What's next?
If you've found a bug or are missing some kind of vocabulary/functionality/etc please contribute or open an issue!