Spark-Redis

A Spark library for Redis

License	BSD 3 Clause
Categories	Redis Data Databases
GroupId	com.redislabs
ArtifactId	spark-redis
Last Version	2.4.1
Release Date	Mar 9, 2020
Type	jar
Description	Spark-Redis A Spark library for Redis
Project URL	http://github.com/RedisLabs/spark-redis
Project Organization	RedisLabs
Source Code Management	https://github.com/RedisLabs/spark-redis

Download spark-redis

Filename	Size
spark-redis-2.4.1.pom
spark-redis-2.4.1.jar	456 KB
spark-redis-2.4.1-sources.jar	35 KB
spark-redis-2.4.1-javadoc.jar	621 KB
spark-redis-2.4.1-jar-with-dependencies.jar	27 MB

How to add to project

Apache Maven

<!-- https://jarcasting.com/artifacts/com.redislabs/spark-redis/ -->
<dependency>
    <groupId>com.redislabs</groupId>
    <artifactId>spark-redis</artifactId>
    <version>2.4.1</version>
</dependency>

Gradle Groovy

// https://jarcasting.com/artifacts/com.redislabs/spark-redis/
implementation 'com.redislabs:spark-redis:2.4.1'

Gradle Kotlin

// https://jarcasting.com/artifacts/com.redislabs/spark-redis/
implementation ("com.redislabs:spark-redis:2.4.1")

Apache Buildr

'com.redislabs:spark-redis:jar:2.4.1'

Apache Ivy

<dependency org="com.redislabs" name="spark-redis" rev="2.4.1">
  <artifact name="spark-redis" type="jar" />
</dependency>

Groovy Grape

@Grapes(
@Grab(group='com.redislabs', module='spark-redis', version='2.4.1')
)

Scala SBT

libraryDependencies += "com.redislabs" % "spark-redis" % "2.4.1"

Leiningen

[com.redislabs/spark-redis "2.4.1"]

Dependencies

compile (5)

Group / Artifact	Type	Version
org.apache.commons : commons-pool2	jar	2.0
redis.clients : jedis	jar	3.2.0
org.scala-lang : scala-library	jar	2.11.12
org.scala-lang : scalap	jar	2.11.12
org.scala-lang : scala-compiler	jar	2.11.12

provided (3)

Group / Artifact	Type	Version
org.apache.spark : spark-core_2.11	jar	2.4.1
org.apache.spark : spark-streaming_2.11	jar	2.4.1
org.apache.spark : spark-sql_2.11	jar	2.4.1

test (1)

Group / Artifact	Type	Version
org.scalatest : scalatest_2.11	jar	2.2.1

Project Modules

There are no modules declared in this project.

Spark-Redis

A library for reading and writing data in Redis using Apache Spark.

Spark-Redis provides access to all of Redis' data structures - String, Hash, List, Set and Sorted Set - from Spark as RDDs. It also supports reading and writing with DataFrames and Spark SQL syntax.

The library can be used both with Redis stand-alone as well as clustered databases. When used with Redis cluster, Spark-Redis is aware of its partitioning scheme and adjusts in response to resharding and node failure events.

Spark-Redis also supports Spark Streaming (DStreams) and Structured Streaming.

Version compatibility and branching

The library has several branches, each corresponds to a different supported Spark version. For example, 'branch-2.3' works with any Spark 2.3.x version. The master branch contains the recent development for the next release.

Spark-Redis	Spark	Redis	Supported Scala Versions
2.4	2.4.x	>=2.9.0	2.11, 2.12
2.3	2.3.x	>=2.9.0	2.11
1.4	1.4.x		2.10

Known limitations

• Java, Python and R API bindings are not provided at this time

Additional considerations

This library is a work in progress so the API may change before the official release.

Documentation

Please make sure you use documentation from the correct branch (2.4, 2.3, etc).

• Getting Started
• RDD
• Dataframe
• Streaming
• Structured Streaming
• Cluster
• Java
• Python
• Configuration
• Dev environment

Contributing

You're encouraged to contribute to the Spark-Redis project.

There are two ways you can do so:

Submit Issues

If you encounter an issue while using the library, please report it via the project's issues tracker.

Author Pull Requests

Code contributions to the Spark-Redis project can be made using pull requests. To submit a pull request:

1. Fork this project.
2. Make and commit your changes.
3. Submit your changes as a pull request.

Redis Labs

Home of Redis

Versions

Version
2.4.1 Mar 9, 2020
2.4.0 May 12, 2019
2.3.1 Mar 25, 2019
2.3.1-RC1 Feb 17, 2019
2.3.1-m3 Nov 22, 2018
2.3.1-M2 Nov 11, 2018
2.3.1-M1 Oct 23, 2018
2.3.0 Aug 14, 2018
1.4.1 Aug 14, 2018