JarCastingIdem
This repository hosts Idem, a Java library that implements a decentralized, k-ordered approach to ID generation using Flake IDs. Idem comes with a couple of building blocks that are used to implement a variety of Flake IDs with differing characteristics. Currently, Idem realizes Flake-64 and Flake-128 IDs.
Why should I bother?
If you are looking for ways to generate unique IDs that exhibit certain properties and require no coordination between individual nodes in your cluster, then you probably hit one of the following problems:
- Generating technical identifiers for database entities hits a bottleneck, since your demand on the write-side of your application is too frequent for any database-centric approach to ID generation to keep up. You need to generate unique IDs and you need it done fast without any coordination required between individual nodes in your cluster.
- Generated IDs should have exhibit some loose form of ordering if they are sorted lexicographically.
- Application-level generated UUIDs of versions that incorporate time are of no use either, since they are susceptible to clock drift.
- Of course you can implement your own library, but have you thought about NTP updates messing with your local time source (e.g. time flows backwards due to adjustments for clock drift)?
Idem implements a fast and reliable way to generate Flake IDs that require no coordination between nodes in your cluster.
Provided ID generators
Idem currently comes with the following ID generators equipped.
| Generator | Width | Target Representation | Comprises |
|---|---|---|---|
Flake64L |
64 | Long |
41-bit timestamp, 10-bit worker ID, 12-bit sequence |
Flake64S |
64 | String (Base62-encoded) |
41-bit timestamp, 10-bit worker ID, 12-bit sequence |
Flake128S |
128 | String (Base62-encoded) |
64-bit timestamp, 48-bit worker ID, 16-bit sequence |
Using Idem
Using the individual Flake ID generators is quite easy. Have a look at the following example which demonstrates how to instantiate a Flake-64 generator that represents generated IDs as java.lang.Long-typed values.
Flake64L flake64 = new Flake64L(
new LinearTimeProvider(),
new StaticWorkerIdProvider("your-application-process"));
Long generatedId = flake64.nextId();
How fast is it?
Idem uses JMH to generate microbenchmarks that exercise the ID generation for the provided Flake IDs. All tests were conducted on a Dell XPS 15" with Intel Core i7-7700HQ CPU @ 2.80 GHz (8 cores). Test were executed with 20 warmup iterations (each one second) before 20 measurement iterations (each one second as well), all single-threaded.
| Benchmark for | Mode | Threads | Mean | Mean Error |
|---|---|---|---|---|
Flake64L |
Throughput | 1 | 2.269.683 op / s | 16.148 op / s |
Flake64S |
Throughput | 1 | 2.322.619 op / s | 12.502 op / s |
Flake128S |
Throughput | 1 | 2.774.614 op / s | 5.874 op / s |
java.util.UUID (*) |
Throughput | 1 | 1.512.316 op / s | 2.363 op / s |
(*) This is simply a call to UUID.randomUUID().toString() which incorporates generation (cf. randomUUID()) and encoding (cf. toString()).
You can run the benchmarks for yourself of course. The module idem-benchmark contains all benchmarks that can be run by selecting the Maven profile benchmark when building the solution.
License
This work is released under the terms of the Apache 2.0 license.
