JarCastingAvro compatibility
A user friendly API for checking for and reporting on Avro schema incompatibilities.
Start using
You can obtain avro-compatibility from Maven Central:
Overview
Although Avro is excellent at determining the compatibility of schemas, it is not very good at reporting the nature and the location of any schema incompatibilities it finds. Rather than supplement or modify any existing Avro APIs (which are already overloaded), avro-compatibility introduces an entirely distinct API that leverages these new features.
Features
Incompatibility type detection
Detects specific incompatibility conditions (via AVRO-1933):
- Type name mismatch.
- Fixed type size mismatch.
- Missing enum symbols.
- Missing field default values.
- Type mismatches.
- Missing union type branches.
Incompatibility location detection
Describes the location in the schema where incompatibilities occur using JSON Pointer (via AVRO-2003). Examples (source schema not shown):
- field type incompatibility:
/fields/0/type - missing union branch:
/fields/1/type/2 - missing enum symbol:
/fields/2/type/symbols - array element type incompatibility:
/fields/3/type/items - fixed size type incompatibility:
/fields/5/type/size - missing default from field:
/fields/3
Descriptive messages
Provides user friendly incompatibility messages such as:
Compatibility type 'CAN_READ' does not hold between schemas, incompatibilities: ['MISSING_UNION_BRANCH: reader union lacking writer type: INT' at '/fields/0/type/2'].
Compatibility type 'CAN_READ' does not hold between one or more schemas because: Schema[1] has incompatibilities: ['READER_FIELD_MISSING_DEFAULT_VALUE: f2' at '/fields/1]'.
Usage
Checking compatibility
With a fluent API:
// 'Backwards'
Compatibility.checkThat(schema2).canRead(schema1);
// 'Backwards, latest of chronology'
Compatibility.checkThat(schema1).canRead().latestOf(schema3, schema2);
// 'Forwards, transitive'
Compatibility.checkThat(schema1).canBeReadBy().all(schema2, schema3);
Alternatively use predefined compatibility checkers:
Compatibility.Mode.CAN_READ_LATEST
.check(schema1, Collections.singletonList(schema3, schema2));
Compatibility.Mode.MUTUAL_READ_WITH_ALL
.check(schema1, Arrays.asList(schema3, schema2));
Compatibility check results
Interrogate the results programmatically with CompatibilityCheckResult and ChronologyCompatibilityCheckResult or simply throw an exception or get a message:
Compatibility.checkThat(schema2).canRead(schema1).throwIfIncompatible();
System.out.println(
Compatibility.checkThat(schema2).canRead(schema1).asMessage()
);
Notes
- The compatibility/evolution rule implementation used by the library supports
aliases; the implementation accessed viaorg.apache.avro.SchemaValidatorBuilderdoes not. Exercise care if migrating from one to the other. Note that this isn't something that we've introduced, Avro just happens to contain two implementations of said rules that unfortunately have subtle differences in behaviour.
Prior art
This project is based on the SchemaCompatibility class from the Avro project. Tests are based on the AvroCompatibilityTest suite from the Confluent/schema-registry project.
Credits
- Created by Elliot West.
- Thanks to @epkanol for the foundational work in AVRO-1933.
- Thanks to @chids for insightful feature suggestions.
- Further thanks to Adrian Woodhead and Dave Maughan.
Legal
This project is available under the Apache 2.0 License. See NOTICE for further information.
Copyright 2017 Expedia Inc.
