flume-header-fields-copy-interceptor

Copies flume event header fields to output JSON extraData field.

License	The Apache License, Version 2.0
GroupId	org.keedio.flume.interceptor.enrichment
ArtifactId	flume-event-header-fields-copy-interceptor
Last Version	0.0.2
Release Date	Dec 2, 2015
Type	jar
Description	flume-header-fields-copy-interceptor Copies flume event header fields to output JSON extraData field.
Project URL	https://github.com/keedio/flume-event-header-fields-copy-interceptor
Source Code Management	https://github.com/keedio/flume-event-header-fields-copy-interceptor

Download flume-event-header-fields-copy-interceptor

Filename	Size
flume-event-header-fields-copy-interceptor-0.0.2.pom
flume-event-header-fields-copy-interceptor-0.0.2.jar	16 KB
flume-event-header-fields-copy-interceptor-0.0.2-sources.jar	1 KB
flume-event-header-fields-copy-interceptor-0.0.2-javadoc.jar	28 KB

How to add to project

Apache Maven

<!-- https://jarcasting.com/artifacts/org.keedio.flume.interceptor.enrichment/flume-event-header-fields-copy-interceptor/ -->
<dependency>
    <groupId>org.keedio.flume.interceptor.enrichment</groupId>
    <artifactId>flume-event-header-fields-copy-interceptor</artifactId>
    <version>0.0.2</version>
</dependency>

Gradle Groovy

// https://jarcasting.com/artifacts/org.keedio.flume.interceptor.enrichment/flume-event-header-fields-copy-interceptor/
implementation 'org.keedio.flume.interceptor.enrichment:flume-event-header-fields-copy-interceptor:0.0.2'

Gradle Kotlin

// https://jarcasting.com/artifacts/org.keedio.flume.interceptor.enrichment/flume-event-header-fields-copy-interceptor/
implementation ("org.keedio.flume.interceptor.enrichment:flume-event-header-fields-copy-interceptor:0.0.2")

Apache Buildr

'org.keedio.flume.interceptor.enrichment:flume-event-header-fields-copy-interceptor:jar:0.0.2'

Apache Ivy

<dependency org="org.keedio.flume.interceptor.enrichment" name="flume-event-header-fields-copy-interceptor" rev="0.0.2">
  <artifact name="flume-event-header-fields-copy-interceptor" type="jar" />
</dependency>

Groovy Grape

@Grapes(
@Grab(group='org.keedio.flume.interceptor.enrichment', module='flume-event-header-fields-copy-interceptor', version='0.0.2')
)

Scala SBT

libraryDependencies += "org.keedio.flume.interceptor.enrichment" % "flume-event-header-fields-copy-interceptor" % "0.0.2"

Leiningen

[org.keedio.flume.interceptor.enrichment/flume-event-header-fields-copy-interceptor "0.0.2"]

Dependencies

compile (1)

Group / Artifact	Type	Version
org.keedio.flume.interceptor.enrichment : flume-enrichment-interceptor	jar	0.0.10

provided (4)

Group / Artifact	Type	Version
org.apache.flume : flume-ng-core	jar	1.5.2
com.googlecode.juniversalchardet : juniversalchardet	jar	1.0.3
org.codehaus.jackson : jackson-core-asl	jar	1.9.13
org.codehaus.jackson : jackson-mapper-asl	jar	1.9.13

test (2)

Group / Artifact	Type	Version
org.testng : testng	jar	6.1.1
org.keedio.flume.interceptor.enrichment : flume-enrichment-interceptor	test-jar	0.0.9-SNAPSHOT

Project Modules

There are no modules declared in this project.

flume event header fields copy interceptor

This Flume interceptor extends Keedio's enrichment interceptor and adds the ability to copy Flume event header fieldsto the enriched extraData message.

Add the following properties to your agent configuration:

To configure your agent to use this interceptor, first configure all the properties documented in Keedio's enrichment interceptor. Add the following properties:

# interceptor
a1.sources.r1.interceptors = i1
a1.sources.r1.interceptors.i1.type = org.keedio.flume.interceptor.enrichment.FieldCopyInterceptor.Builder

## other properties
...

# Fields to copy from the Flume event header to the extraData field
a1.sources.r1.interceptors.i1.header.fields.to.copy.header1 = header1
a1.sources.r1.interceptors.i1.header.fields.to.copy.header2 = header2
a1.sources.r1.interceptors.i1.header.fields.to.copy.header3 = differentName3
# Optional reverse.copy strategy (false by default)
a1.sources.r1.interceptors.i1.reverse.copy = false

This will produce an event whose body contains an enriched message in which the extraData field contains the values for headers header1 and header2 coming in the input flume event. Also, it will contain the value for header header3 but with the name differentName3 in the extraData field.

If the specified headers do not exist, the extraData field won't be modified.

If the parameter reverse.copy contains true (false by default) the copy strategy will be reversed: The property keys header1, header2 and header3 will refer to extraData fields and values header1, header2 and differentName3 will refer to header fields where the values should be copied to.

keedio

Versions

Version
0.0.2 Dec 2, 2015