JarCastingJavaTest
|
|
Fair Warning: The entirety of this project should currently be considered alpha and subject to change. I am sure I have not got the API or library right on the first try so there may be breaking changes in the future. |
Overview
An opinionated, functional test framework written in pure Java aiming to leverage newer features of the Java language and give power back to the test writer. Based on these ideas:
-
If you know how to write a Java application, I think you should automatically know how to write the tests, therefore JavaTest only introduces a few new functions and types to the language, there is no new syntax.
-
Instead of being Annotation and Exception driven, JavaTest is Function and Value driven, lending itself to composability of tests and assertions.
-
JavaTests contains no reflection magic by default. A more declarative approach to testing is taken where setting up and customising tests is up to the writer.
-
A test should ideally only test one thing and should certainly test at least one thing. Since all tests must return an assertion value the compiler will enforce that all tests only test one result.
-
Tests should be easy to understand and enjoyable to write, after all we all spend a lot of our time working on them :D
Quick Start
I’m new to Java
Expand
Download the latest jar artifact of JavaTest Core from the release page. Then create these files in your project directory:
-
foo/Calculator.javaThis is the System Under Test representing the source code for your application (in this case a calculator that can add integers)
package foo; public class Calculator { public static int add(int a, int b) { // We are intentionally making this function return the wrong value. // This is so you can see the tests fail, then fix the function and see them pass return a + b + 10; } }
-
foo/Tests.javaThis file is a java executable containing tests for our SUT, it exists in the same package so there is no need to
import foo.Calculator;.This example defines two simple tests, one is testing that
1 + 1 = 2by simply using the java+function and the other test checks our calculator gets the same result. We then invoke therunTestsfunction to run our tests and check if they passed.package foo; import java.util.List; import static io.github.olib963.javatest.JavaTest.*; public class Tests { public static void main(String... args) { var result = runTests(List.of( test("Addition", () -> that(1 + 1 == 2, "Math still works, one add one is still two")), test("Calculator Addition", () -> { var one = 1; var expected = 2; var additionResult = Calculator.add(1, 1); var description = "Expected %s add %s to be %s (Calculator returned %s)"; var formatted = String.format(description, one, one, expected, additionResult); return that(additionResult == expected, formatted); }))); if (!result.succeeded) { throw new RuntimeException("Tests failed!"); } System.out.println("Tests passed"); } }
Note how for the second test the assertion description includes all the information required to tell is what has gone wrong if our tests are failing.
You can then run from the commandline:
# Compile both Java classes ensuring JavaTest and the current directory are both on the class path javac -cp "/absolute/path/to/javatest/jar:." foo/Calculator.java foo/Tests.java # Run the "Tests" executable ensuring JavaTest and the current directory are both on the class path java -cp "/absolute/path/to/javatest/jar:." foo.Tests
These tests should currently fail with a nice error message. You should be able to fix the calculator and see your tests pass.
Notes:
-
You will need to use
;to separate classpath entries instead of:on windows machines -
You will need to include at least the JavaTest jar and the current directory (
.) on the classpath in order for this to work, if you are using java classes from any other jars/directories you will need to also ensure they are on the classpath.
You should be able to explore the Core Library and get familiar with testing your code by running them from an executable.
I know Java pretty well
Expand
An example of a test entry point:
import io.github.olib963.javatest.*;
import io.github.olib963.javatest.fixtures.Fixtures;
import io.github.olib963.javatest.javafire.TestRunners;
import java.util.Collection;
import java.util.List;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import static io.github.olib963.javatest.JavaTest.*;
public class MyRunners implements TestRunners {
@Override
public Collection<TestRunner> runners() {
// Define a runner for unit tests in parallel
List<Testable> tests = List.of(new MyFirstUnitTestSuite(), new MySecondUnitTestSuite());
var unitTests = lazyTestableRunner(tests.parallelStream());
// Define integration tests with an executor fixture
var executorDefinition = Fixtures.definitionFromThrowingFunctions(
Executors::newSingleThreadExecutor, ExecutorService::shutdown);
var integrationTests = Fixtures.fixtureRunner("executor",
executorDefinition,
es -> testableRunner(new MyIntegrationTestSuite(es)));
// Run both
return List.of(unitTests, integrationTests);
}
}
An example of a test suite:
import io.github.olib963.javatest.TestSuiteClass;
import io.github.olib963.javatest.Testable;
import java.util.Collection;
import java.util.List;
import static io.github.olib963.javatest.JavaTest.test;
import static io.github.olib963.javatest.matchers.CollectionMatchers.contains;
import static io.github.olib963.javatest.matchers.Matcher.that;
import static io.github.olib963.javatest.matchers.StringMatchers.containsString;
public class MyFirstUnitTestSuite implements TestSuiteClass {
@Override
public Collection<Testable> testables() {
return List.of(
test("List contains", () -> that(List.of(1,2,3), contains(2))),
test("Messaging", () -> {
var myObject = new MyBusinessMessageObject();
var message = myObject.createMessageFor(50);
return that(message, containsString("integer 50"));
})
);
}
}
These tests can be run in a few different ways, look into Running JavaTest to find the way that works best for you.
The Core library section explains the fundamentals of how these tests are defined. Functionality from the Fixtures and Matchers modules are used in this example, explore the Module List to see if there are any extensions that fit your needs.
If there is something you cannot achieve with the existing functionality please open an issue describing what you want to do :D
Contents
Core Library
-
JavaTest: the entrypoint class. It contains the mainrunfunction as well as static factory functions to define tests -
TestRunner: implements the running of tests and returnsTestResults -
Testable: wrapper around one of:-
Test: a named instance of a test, each test must return anAssertion -
TestSuite: a named logical collection ofTestables
-
-
Assertion: represents the expected state at the end of a test -
TestResults: represents the combined result of your tests -
RunConfiguration: An object that contains configuration for a specific run of JavaTest. Currently this just contains side-effecting observers.
Creating Assertions
The core Assertions are created simply from boolean expressions and a string description. If the boolean condition is true then the Assertion is said to "hold".
import io.github.olib963.javatest.Assertion;
import static io.github.olib963.javatest.JavaTest.that;
public class MyAssertions {
Assertion simpleAssertion = that(1 + 1 == 2, "Expected one add one to be two");
public Assertion multilineFormattedAssertion() {
var two = 2;
var ten = 10;
var twenty = 20;
var expected = String.format("Expected %d times %d to be %d", two, ten, twenty);
return that(two * ten == twenty, expected);
}
}
Composing Assertions
Assertions can be composed using the and, or and xor default methods. These are all examples of composed Assertions that hold (i.e. will pass tests):
import io.github.olib963.javatest.Assertion;
import static io.github.olib963.javatest.JavaTest.*;
public class MyComposedAssertions {
Assertion orAssertion = that(1 + 1 == 3, "Expected one add one to be three")
.or(that(2 + 2 == 4, "Expected two add two to be four"));
Assertion andAssertion = that(1 + 1 == 2, "Expected one add one to be two").and(orAssertion);
Assertion xorAssertion = that(true, "Expected to hold").xor(that(false, "Expected not to hold"));
}
In the case where you have many assertions (perhaps a large list) I would suggest using the all assertion functionality for a better output:
import io.github.olib963.javatest.Assertion;
import static io.github.olib963.javatest.JavaTest.*;
public class MyAllAssertions {
Assertion allAssertion = all(
that(1 + 1 == 2, "Expected one add one to be two"),
that(2 + 2 == 4, "Expected two add two to be four"),
that(3 + 3 == 6, "Expected three add three to be six"),
that(4 + 4 == 8, "Expected four add four to be eight"),
that(5 + 5 == 10, "Expected five add five to be ten")
);
}
Test Definitions
A JavaTest Test is defined by:
-
A name
-
A
Supplierof anAssertion
import io.github.olib963.javatest.Testable.Test;
import static io.github.olib963.javatest.JavaTest.*;
public class MyTests {
Test myFirstTest = test("Simple Test", () -> that(true, "Expected test to pass"));
}
The test will fail if the Supplier throws any exception at all. Please see Notes for information on AssertionErrors.
Test Suites
You can group your Tests into logical units using TestSuites.
import io.github.olib963.javatest.Testable.TestSuite;
import java.util.List;
import static io.github.olib963.javatest.JavaTest.*;
public class MyFirstTestSuite {
public static TestSuite mySuite() {
return suite("MyTests", List.of(
test("Simple Test", () -> that(true, "Expected test to pass"))
));
}
}
Suite Classes
A common use case will be to use a class to store your tests, to do this simply implement TestSuiteClass. Your class will then be able to be used anywhere you would use a Testable e.g. adding to another TestSuite or passing to JavaTests run functions. The name of the suite will be the name of the class.
import io.github.olib963.javatest.TestSuiteClass;
import io.github.olib963.javatest.Testable;
import java.util.Collection;
import java.util.List;
import static io.github.olib963.javatest.JavaTest.*;
public class ClassAsSuite implements TestSuiteClass {
@Override
public Collection<Testable> testables() {
return List.of(
test("Simple Test", () -> that(true, "Expected test to pass"))
);
}
}
Suite Nesting
TestSuites contain Testables not Tests and thus can contain other TestSuites.
import io.github.olib963.javatest.Testable.TestSuite;
import java.util.List;
import static io.github.olib963.javatest.JavaTest.*;
public class SuiteOfSuites {
// A suite composed of one test and two suites
public static TestSuite compositeSuite() {
return suite("MyComposedTests",
List.of(
test("Simple Test", () -> that(true, "Expected test to pass")),
MyFirstTestSuite.mySuite(),
new ClassAsSuite()
));
}
}
Pending Tests
Sometimes it will be useful to define a bunch of Test cases ahead of implementing them, this is where pending Assertions come in. They will not fail your build but will logged in a different colour than successes/failures if using the coloured logger. You can optionally provide a reason this Test has not yet been written.
import io.github.olib963.javatest.TestSuiteClass;
import io.github.olib963.javatest.Testable;
import java.util.Collection;
import java.util.List;
import static io.github.olib963.javatest.JavaTest.*;
public class MyPendingTests implements TestSuiteClass {
@Override
public Collection<Testable> testables() {
return List.of(
test("Addition", () -> that(1 + 1 == 2, "Expected one add one to be two")),
test("Multiplication", () -> pending()),
test("Division by Zero",
() -> pending("I am not yet sure if this should throw an exception or return a failure value"))
);
}
}
Test Runners
The main TestRunner included in the core is created from a Collection<Testable>.
import java.util.Collections;
import java.util.List;
import io.github.olib963.javatest.*;
import static io.github.olib963.javatest.JavaTest.*;
public class MyRunners {
public TestRunner singleTestRunner = testableRunner(List.of(
test("Simple test", () -> pending())));
public TestRunner suiteTests = testableRunner(
List.of(MyFirstTestSuite.mySuite(), new ClassAsSuite())
);
}
Laziness
It is possible to create a lazy test runner from a Stream<Testable>. This runner however is not referentially transparent or reusable so must be used with care. This might be useful if you have a very large collection of tests and you want to lazily instantiate the different suites.
import java.util.stream.Stream;
import io.github.olib963.javatest.*;
import static io.github.olib963.javatest.JavaTest.*;
public class LazyRunners {
private Stream<Testable.Test> oneHundredLazyTests =
Stream.generate(() -> test("Lazy test", () -> pending()))
.limit(100);
public TestRunner lazyRunner = lazyTestableRunner(oneHundredLazyTests);
}
Other TestRunner implementations are available in the other modules.
Run Configuration
You can optionally add a collection of TestCompletionObservers and TestRunCompletionObservers to the run of JavaTest. By default a logging observer is passed that logs each test result with a colour corresponding to the state of the test (green for passing, red for failing and yellow for pending) and a logging observer for the completion of the run that logs the total failed, passing and pending results.
import io.github.olib963.javatest.RunConfiguration;
import io.github.olib963.javatest.TestCompletionObserver;
public class RunConfigurations {
// Configuration that logs individual tests in colour and the full results of the run
public RunConfiguration defaultConfig = RunConfiguration.defaultConfig();
// Configuration that logs individual tests with no colour and logs a warning
// if there are any pending tests at all in the run.
public RunConfiguration customConfig = RunConfiguration.empty()
.addTestObserver(TestCompletionObserver.plainLogger())
.addRunObserver(results -> {
if (results.pendingCount != 0) {
System.err.println("!!You still have unwritten tests!!");
}
});
}
Core library maven dependency
<dependency>
<groupId>io.github.olib963</groupId>
<artifactId>javatest-core</artifactId>
<version>${javatest.version}</version>
<scope>test</scope>
</dependency>
Running JavaTest
In Java
To run JavaTest simply pass your TestRunner instances to the JavaTest.run() function and handle the result how you see fit. There is a convenience function runTests defined to just run a Collection<Testable> using the default CollectionRunner:
import io.github.olib963.javatest.*;
import java.util.List;
import static io.github.olib963.javatest.JavaTest.*;
public class MyEntrypoint {
public static void main(String... args) {
var results = runTests(List.of(
test("Addition", () -> that(1 + 1 == 2, "Expected one add one to be two")),
test("String lower case", () ->
that("HELLO".toLowerCase().equals("hello"), "Expected lowercase 'HELLO' to be 'hello'"))
));
var customResults = run(new MyCustomRunner());
if(results.succeeded && customResults.succeeded) {
System.out.println("Yay tests passed! :)");
} else {
throw new RuntimeException("Boo tests failed! :(");
}
}
}
With JavaFire Maven plugin
If you are using maven you can add the JavaFire maven plugin to your pom to run your tests during mavens test phase. By default this will use the Reflection Module to run any instances of TestSuiteClass or TestRunners in your test source directory that have a public zero argument constructor.
<plugin>
<groupId>io.github.olib963</groupId>
<artifactId>javafire-maven-plugin</artifactId>
<version>${javatest.version}</version>
<executions>
<execution>
<id>test</id>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
Alternatively you can run tests defined by a single TestRunners class. Your TestRunners class must have a public zero argument constructor. This is achieved by passing the class name to the plugin configuration for example:
package my.awesome.app;
import io.github.olib963.javatest.*;
import io.github.olib963.javatest.fixtures.Fixtures;
import io.github.olib963.javatest.javafire.TestRunners;
import java.util.Collection;
import java.util.List;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import static io.github.olib963.javatest.JavaTest.*;
public class MyRunners implements TestRunners {
@Override
public Collection<TestRunner> runners() {
// Define a runner for unit tests in parallel
List<Testable> tests = List.of(new MyFirstUnitTestSuite(), new MySecondUnitTestSuite());
var unitTests = lazyTestableRunner(tests.parallelStream());
// Define integration tests with an executor fixture
var executorDefinition = Fixtures.definitionFromThrowingFunctions(
Executors::newSingleThreadExecutor, ExecutorService::shutdown);
var integrationTests = Fixtures.fixtureRunner("executor",
executorDefinition,
es -> testableRunner(new MyIntegrationTestSuite(es)));
// Run both
return List.of(unitTests, integrationTests);
}
}
In pom.xml:
<plugin>
<groupId>io.github.olib963</groupId>
<artifactId>javafire-maven-plugin</artifactId>
<version>${javatest.version}</version>
<configuration>
<testRunners>my.awesome.app.MyRunners</testRunners>
</configuration>
<executions>
<execution>
<id>test</id>
<goals>
<goal>test</goal>
</goals>
</execution>
</executions>
</plugin>
You can override the testRunners class being used by setting the maven property e.g. mvn -Djavafire.testRunners=com.my.app.OtherTests test.
Run Configuration
You can provide your own RunConfiguration class within the plugin configuration. If you create a class with a public zero argument constructor that implements RunConfigurationProvider such as:
package my.awesome.app;
import io.github.olib963.javatest.RunConfiguration;
import io.github.olib963.javatest.TestCompletionObserver;
import io.github.olib963.javatest.TestResult;
import io.github.olib963.javatest.TestRunCompletionObserver;
import io.github.olib963.javatest.javafire.RunConfigurationProvider;
/**
* My config provider only logs failed tests and the full run results
*/
public class MyConfigProvider implements RunConfigurationProvider {
@Override
public RunConfiguration config() {
var loggingObserver = TestCompletionObserver.colourLogger();
TestCompletionObserver onlyLogFailures = result -> {
if(testFailed(result)) {
loggingObserver.onTestCompletion(result);
}
};
return RunConfiguration.empty()
.addRunObserver(TestRunCompletionObserver.logger())
.addTestObserver(onlyLogFailures);
}
private boolean testFailed(TestResult result) {
return result.match(
suiteResult -> suiteResult.results().anyMatch(this::testFailed),
singleTestResult -> !singleTestResult.result.holds
);
}
}
then you can set the configuration option in the plugin:
<configuration>
<runConfigurationProvider>my.awesome.app.MyConfigProvider</runConfigurationProvider>
</configuration>
You can also override the provider class on the commandline by setting the maven property e.g. mvn -Djavafire.runConfigurationProvider=com.my.app.ConfigProvider test.
JShell
Since JavaTest is built on pure Java it plays quite nicely with the REPL. This startup script may be useful to you:
/env -class-path ${ABSOLUTE_PATH_TO_JAVATEST_JAR}
import io.github.olib963.javatest.*;
import static io.github.olib963.javatest.JavaTest.*;
TestResults runTest(CheckedSupplier<Assertion> testFn) {
return runTests(test("JShell test", testFn));
}Then you can run:
~$ jshell --startup DEFAULT --startup javatest-startup-script.jsh
| Welcome to JShell -- Version 11.0.2
| For an introduction type: /help intro
jshell> var results = runTest(() -> that(true, "JavaTest works in the shell!"))
JShell test:
JavaTest works in the shell!
Ran a total of 1 tests.
1 succeeded
0 failed
0 were pending
results ==> TestResults{succeeded=true, successCount=1, failu ... logs=[]'}, testLogs=[]}]}
jshell> var results2 = runTest(() -> that(1 + 1 == 3, "One add One is Three"))
JShell test:
One add One is Three
Ran a total of 1 tests.
0 succeeded
1 failed
0 were pending
results2 ==> TestResults{succeeded=false, successCount=0, fail ... logs=[]'}, testLogs=[]}]}
jshell> var allResults = results.combine(results2)
allResults ==> TestResults{succeeded=false, successCount=1, fail ... logs=[]'}, testLogs=[]}]}
jshell> allResults.succeeded
$7 ==> false
jshell> allResults.failureCount
$8 ==> 1
jshell> allResults.allResults().collect(java.util.stream.Collectors.toList())
$9 ==> [SingleTestResult{name='JShell test', result=AssertionResult{holds=true, pending=false, description='JavaTest works in the shell!, logs=[]'}, testLogs=[]}, SingleTestResult{name='JShell test', result=AssertionResult{holds=false, pending=false, description='One add One is Three, logs=[]'}, testLogs=[]}]
jshell> /exit
| Goodbye
Maven Integration
If you are using maven there is a bill of materials you can import to manage the versions of dependencies. You can add this to your pom by doing the following?
<dependencyManagement>
<dependencies>
<dependency>
<groupId>io.github.olib963</groupId>
<artifactId>javatest-bom</artifactId>
<version>${javatest.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>io.github.olib963</groupId>
<artifactId>javatest-core</artifactId>
</dependency>
<dependency>
<groupId>io.github.olib963</groupId>
<artifactId>javatest-matchers</artifactId>
</dependency>
<!-- ... More dependencies -->
</dependencies>
Module List
JavaTest is built on a simple functional core and functionality is expanded on by several modules found here:
Functional Interfaces
Where possible interfaces are @FunctionalInterfaces so can be replaced with lambdas when you feel it fits. This is true for:
-
Assertions
import io.github.olib963.javatest.Assertion;
import io.github.olib963.javatest.AssertionResult;
public class FunctionalAssertions {
// Create an assertion that always fails
public static final Assertion ALWAYS_FAILING =
() -> AssertionResult.failure("Whoops");
// Do not attempt to run an assertion if the variable is not set
public Assertion ensureEnvironmentVariableSet(String variable, Assertion assertion) {
return () -> {
if (System.getenv(variable) == null) {
return AssertionResult.failure("You must set the environment variable " + variable);
} else {
return assertion.run();
}
};
}
}
-
TestRunners
import io.github.olib963.javatest.TestResults;
import io.github.olib963.javatest.TestRunner;
import java.time.DayOfWeek;
import java.time.LocalDate;
import java.util.Set;
public class FunctionalRunners {
// The simplest test runner: runs nothing and returns no results
public static final TestRunner EMPTY_RUNNER = config -> TestResults.empty();
private static final Set<DayOfWeek> WEEKEND =
Set.of(DayOfWeek.SATURDAY, DayOfWeek.SUNDAY);
// Wrap another test runner such that it will not run anything on the weekend
public TestRunner onlyRunOnWeekDays(TestRunner runner) {
var today = LocalDate.now();
return config -> WEEKEND.contains(today.getDayOfWeek()) ?
EMPTY_RUNNER.run(config) : runner.run(config);
}
}
-
TestCompletionObservers andTestRunCompletionObservers
import io.github.olib963.javatest.TestCompletionObserver;
import io.github.olib963.javatest.TestRunCompletionObserver;
public class FunctionalObservers {
// Log a message to yourself to remind you that you have still have tests to write
public TestRunCompletionObserver personalWarning = result -> {
if (result.pendingCount != 0) {
System.out.println("\n\n\n!!You still have unwritten tests!!\n\n\n");
}
};
// Replace logging with simple "X completed" log
public TestCompletionObserver simpleLog = result -> {
var log = result.match(
suiteResult -> suiteResult.suiteName + "completed",
singleTestResult -> singleTestResult.name + "completed"
);
System.out.println(log);
};
}
In the cases where this is not possible (e.g. Matchers or FixtureDefinitions) static factories will be provided to construct an instance of the interface from appropriate functions.
Feedback
Any feedback/constructive criticism is appreciated. Please open an issue if you have any suggestions.
Notes
-
Documentation is built using asciidoctor such that all documentation snippits can be both compile time checked and tested.
-
If a test throws an
AssertionErrorinstead of returning anAssertionthe library will ignore the error message and instead tell you to "return an Assertion". This will stop you from trying to add in extra assertions using something like the JUnitassertXfunctions in the middle of your test. -
Currently there is no way to programmatically find the source location of a test/suite. This is a challenge I am going to attempt to tackle once I implement an IDE plugin.