JarCasting
JarCasting JarCasting

io.thill.kafkacap:kafkacap-aeron

Reliably Capture Messages to Kafka

License

License
Categories

Categories

Aeron Application Layer Libs Messaging
GroupId

GroupId

io.thill.kafkacap
ArtifactId

ArtifactId

kafkacap-aeron
Last Version

Last Version

0.1.1
Release Date

Release Date
Type

Type

jar
Description

Description

Reliably Capture Messages to Kafka

Download kafkacap-aeron

Filename Size
kafkacap-aeron-0.1.1.pom
kafkacap-aeron-0.1.1.jar 10 KB
kafkacap-aeron-0.1.1-sources.jar 7 KB
kafkacap-aeron-0.1.1-javadoc.jar 52 KB
Browse

How to add to project

<!-- https://jarcasting.com/artifacts/io.thill.kafkacap/kafkacap-aeron/ -->
<dependency>
    <groupId>io.thill.kafkacap</groupId>
    <artifactId>kafkacap-aeron</artifactId>
    <version>0.1.1</version>
</dependency>
// https://jarcasting.com/artifacts/io.thill.kafkacap/kafkacap-aeron/
implementation 'io.thill.kafkacap:kafkacap-aeron:0.1.1'
// https://jarcasting.com/artifacts/io.thill.kafkacap/kafkacap-aeron/
implementation ("io.thill.kafkacap:kafkacap-aeron:0.1.1")
'io.thill.kafkacap:kafkacap-aeron:jar:0.1.1'
<dependency org="io.thill.kafkacap" name="kafkacap-aeron" rev="0.1.1">
  <artifact name="kafkacap-aeron" type="jar" />
</dependency>
@Grapes(
@Grab(group='io.thill.kafkacap', module='kafkacap-aeron', version='0.1.1')
)
libraryDependencies += "io.thill.kafkacap" % "kafkacap-aeron" % "0.1.1"
[io.thill.kafkacap/kafkacap-aeron "0.1.1"]

Dependencies

compile (3)

Group / Artifact Type Version
io.thill.kafkacap : kafkacap-core jar 0.1.1
org.slf4j : slf4j-simple jar 1.7.25
io.aeron : aeron-client jar 1.19.1

test (3)

Group / Artifact Type Version
junit : junit jar 4.12
io.thill.kafkalite : kafka-lite jar 0.9.1
io.aeron : aeron-driver jar 1.19.1

Project Modules

There are no modules declared in this project.

KafkaCap

Reliably Capture Messages to Kafka

Architecture

  • 2 or more Capture Devices write to separate Kafka Topics
  • A Deduplicator process listens to the Capture Device topics and writes unique messages to a single Kafka Topic
  • Multiple Deduplicator processes with the same Kafka Consumer group.id run to provide fault-tolerance

Architecture

Core

Capture Device

"Bring Your Own Receiver" - An abstract CaptureDevice class allows simple plug-and-play

Deduplicator

Instantiate a Deduplicator with a custom DedupStrategy to generate a unified outbound topic. Downstream consumers can treat the outbound topic as any normal Kafka topic, while having high confidence that all messages are captured and appropriately ordered.

Documentation

Full Details: Core

Provided Implementations

Capture Devices

Versions

Version
0.1.1
0.1.0