JarCastingDropwizard DB Sharding Bundle
This library adds support for Database sharding in Dropwizard based applications. Make sure you're familiar with Dropwizard, dependency injection framework like Guice and concepts like ThreadLocals before going ahead.
License: Apache 2.0
Stable version: 0.2.18
Why this library?
- Traditionally in Dropwizard, to open a database transaction
@UnitOfWorkis put on every Jersey-resource method. In case@UnitOfWorkis required at places, other than Jersey-resources a more verbose approach is required as mentiond here. - Nested
@UnitOfWorkdon't work in Dropwizard, which are some times needed in a sharding use-case. - Dropwizard doesn't support Hibernate's multi-tenancy.
- The original library from where this library has been forked, advocates that database-transaction should only be managed at DAO layer, while it may work in their use-case, for Cleartax being a finance system, clean rollback of transaction (which may include multiple entities in some cases) is very important.
This library solves above problems by:
- Using Guice method interceptors to seamlessly use
@UnitOfWorkin anywhere in the code. Assumption: Methods that are annotated with@UnitOfWorkaren't private and the class must be created via Guice dependency injection. - Handle nested
@UnitOfWorkwithin the same thread. How? - Uses Hibernate's multi-tenancy support and integrates it with Dropwizard.
How to Use
Terminology
- Shard/Tenant mean the same thing, which is the physical database.
- Shard-id/tenant-id also mean the same thing, which is the id of the physical database. (Refer point no. 2 in High-level section)
- Shard-key is the ID on which data is sharded, e.g. if you're sharding by user-id, then user-id becomes your shard-key.
- Bucket is an intermediate virtual-shard to which shard-key gets mapped to.
- Bucket-id gets mapped to the shard-id.
High level
- Include as dependency:
<dependency>
<groupId>in.cleartax.dropwizard</groupId>
<artifactId>sharding-core</artifactId>
<version>0.2.8</version>
</dependency>
for liquibase migration support, you can also include:
<dependency>
<groupId>in.cleartax.dropwizard</groupId>
<artifactId>sharding-migrations</artifactId>
<version>0.2.8</version>
</dependency>
-
Update your Dropwizard YML config to declare all the hosts as described here.
-
Understand Guice's AbstractModule and @Provides.
-
Define all the dependencies in an extension to
AbstractModulethat binds all your classes. e.g. refer this. -
Register your module in your Dropwizard application as described here.
Low level
Consider this method which is annotated with @UnitOfWork.
UnitOfWorkInterceptor by using Guice's AOP would intercept the method call and figure out the shard-id/tenant-id then initiate the transaction. For this to work, you would need to do:
Map all the shard-keys to bucket
UnitOfWorkInterceptor calls the implementation of ShardKeyProvider to map the shard-key to a bucket.
Map all the buckets to shards
UnitOfWorkInterceptor calls the implementation of BucketResolver to figure out shard-id. Refer DbBasedShardResolver to understand one example use-case. In this example, refer this SQL script where the mappings are done.
Connect to right shard, by
-
Setup shard-key for every incoming HTTP request - Refer ShardKeyFeature in the example project. Note:
ShardKeyProviderin the example is bounded to it's implementation in the Guice module described earlier. -
Setup shard-key manually - Get an instance of
ShardKeyProviderand then do:
try {
shardKeyProvider.setKey("shard-key")
// Call your method which is annotated with @UnitOfWork
} finally {
shardKeyProvider.clear();
}
Example use-case: In case you're invoking your code outside of HTTP layer, or you're creating a child-thread which may not have all the context of parent.
- Connect to a shard by explicitly mentioning shard-id
try {
DelegatingTenantResolver.getInstance().setDelegate(new ConstTenantIdentifierResolver("your shard-id/tenant-id"));
// Call your method which is annotated with @UnitOfWork
} finally {
if (DelegatingTenantResolver.getInstance().hasTenantIdentifier()) {
DelegatingTenantResolver.getInstance().setDelegate(null);
}
}
Example use-case: This might be useful in case you're aggregating data from across all the shards
- Connect to a shard without using @UnitOfWork
This is not recommended because then you'll need hibernate specific objects in your business code. In case required, you can instantiate TransactionRunner and use it as mentioned here
Note: In case you don't need sharding but still need flexibility of using @UnitOfWork outside of resources and ability to use nested @UnitOfWork you can still do so. Refer these tests to understand the use-case.
