Infinispan Hadoop Parent

Parent for all Infinispan Hadoop modules

License	Apache License
Categories	Infinispan Data Caching
GroupId	org.infinispan.hadoop
ArtifactId	parent
Last Version	0.4
Release Date	Feb 5, 2019
Type	pom
Description	Infinispan Hadoop Parent Parent for all Infinispan Hadoop modules
Project URL	http://infinispan.org
Project Organization	JBoss, a division of Red Hat
Source Code Management	https://github.com/infinispan/infinispan-hadoop

Download parent

Filename	Size
parent-0.4.pom	9 KB

How to add to project

Apache Maven

<!-- https://jarcasting.com/artifacts/org.infinispan.hadoop/parent/ -->
<dependency>
    <groupId>org.infinispan.hadoop</groupId>
    <artifactId>parent</artifactId>
    <version>0.4</version>
    <type>pom</type>
</dependency>

Gradle Groovy

// https://jarcasting.com/artifacts/org.infinispan.hadoop/parent/
implementation 'org.infinispan.hadoop:parent:0.4'

Gradle Kotlin

// https://jarcasting.com/artifacts/org.infinispan.hadoop/parent/
implementation ("org.infinispan.hadoop:parent:0.4")

Apache Buildr

'org.infinispan.hadoop:parent:pom:0.4'

Apache Ivy

<dependency org="org.infinispan.hadoop" name="parent" rev="0.4">
  <artifact name="parent" type="pom" />
</dependency>

Groovy Grape

@Grapes(
@Grab(group='org.infinispan.hadoop', module='parent', version='0.4')
)

Scala SBT

libraryDependencies += "org.infinispan.hadoop" % "parent" % "0.4"

Leiningen

[org.infinispan.hadoop/parent "0.4"]

Dependencies

There are no dependencies for this project. It is a standalone project that does not depend on any other jars.

Project Modules

There are no modules declared in this project.

Infinispan Hadoop

Integrations with Apache Hadoop and related frameworks.

Compatibility

Version	Infinispan	Hadoop	Java
0.1	8.0.x	2.x	8
0.2	8.2.x	2.x	8
0.3	9.4.x	2.x 3.x	8
0.4	9.4.x	2.x 3.x	8

InfinispanInputFormat and InfinispanOutputFormat

Implementation of Hadoop InputFormat and OutputFormat that allows reading and writing data to Infinispan Server with best data locality. Partitions are generated based on segment ownership and allows processing of data in a cache using multiple splits in parallel.

Maven Coordinates

 <dependency>  
    <groupId>org.infinispan.hadoop</groupId>  
    <artifactId>infinispan-hadoop-core</artifactId>  
    <version>0.4</version>
 </dependency>  

Sample usage with Hadoop YARN mapreduce application:

import org.infinispan.hadoop.*;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.mapreduce.Job;

Configuration configuration = new Configuration();
String hosts = "172.17.0.2:11222;172.17.0.3:11222";

// Configures input/output caches
configuration.set(InfinispanConfiguration.INPUT_REMOTE_CACHE_SERVER_LIST, hosts);
configuration.set(InfinispanConfiguration.OUTPUT_REMOTE_CACHE_SERVER_LIST, hosts);

configuration.set(InfinispanConfiguration.INPUT_REMOTE_CACHE_NAME, "map-reduce-in");
configuration.set(InfinispanConfiguration.OUTPUT_REMOTE_CACHE_NAME, "map-reduce-out");

Job job = Job.getInstance(configuration, "Infinispan job");

// Map and Reduce implementation
job.setMapperClass(MapClass.class);
job.setReducerClass(ReduceClass.class);

job.setInputFormatClass(InfinispanInputFormat.class);
job.setOutputFormatClass(InfinispanOutputFormat.class);

Supported Configurations:

Name	Description	Default
hadoop.ispn.input.filter.factory	The name of the filter factory deployed on the server to pre-filter data before reading	null (no filtering)
hadoop.ispn.input.cache.name	The name of cache where data will be read from	"default"
hadoop.ispn.input.read.batch	Batch size when reading from the cache	5000
hadoop.ispn.output.write.batch	Batch size when writing to the cache	500
hadoop.ispn.input.remote.cache.servers	List of servers of the input cache, in the format host1:port1;host2:port2	localhost:11222
hadoop.ispn.output.cache.name	The name of cache where job results will be written to	"default"
hadoop.ispn.output.remote.cache.servers	List of servers of the output cache, in the format host1:port1;host2:port2
hadoop.ispn.input.converter	Class name with an implementation of org.infinispan.hadoop.KeyValueConverter, applied after reading from the cache	null (no converting)
hadoop.ispn.output.converter	Class name with an implementation of org.infinispan.hadoop.KeyValueConverter, applied before writing	null (no converting)

Demos

Refer to https://github.com/infinispan/infinispan-hadoop/tree/master/samples/

Releasing

The $MAVEN_HOME/conf/settings.xml must contain credentials for the release repository. Add the following section in <servers>:

<server>
   <id>jboss-snapshots-repository</id>
   <username>RELEASE_USER</username>
   <password>RELEASE_PASS</password>
</server>
<server>
   <id>jboss-releases-repository</id>
   <username>RELEASE_USER</username>
   <password>RELEASE_PASS</password>
</server>

To release:

mvn release:prepare release:perform -B

Infinispan

Infinispan is a distributed in-memory key/value data store with optional schema, available under the Apache License 2.0.

Versions

Version
0.4 Feb 5, 2019
0.3 Oct 29, 2018
0.2 Mar 9, 2016
0.1 Sep 16, 2015