spark-exasol-connector

The Spark Exasol Connector

License	Apache License 2.0
GroupId	com.exasol
ArtifactId	spark-connector_2.11
Last Version	0.3.2
Release Date	Oct 1, 2020
Type	jar
Description	spark-exasol-connector The Spark Exasol Connector
Project URL	https://github.com/EXASOL/spark-exasol-connector
Project Organization	com.exasol
Source Code Management	https://github.com/EXASOL/spark-exasol-connector

Download spark-connector_2.11

Filename	Size
spark-connector_2.11-0.3.2.pom
spark-connector_2.11-0.3.2.jar	157 KB
spark-connector_2.11-0.3.2-sources.jar	19 KB
spark-connector_2.11-0.3.2-javadoc.jar	411 KB

How to add to project

Apache Maven

<!-- https://jarcasting.com/artifacts/com.exasol/spark-connector_2.11/ -->
<dependency>
    <groupId>com.exasol</groupId>
    <artifactId>spark-connector_2.11</artifactId>
    <version>0.3.2</version>
</dependency>

Gradle Groovy

// https://jarcasting.com/artifacts/com.exasol/spark-connector_2.11/
implementation 'com.exasol:spark-connector_2.11:0.3.2'

Gradle Kotlin

// https://jarcasting.com/artifacts/com.exasol/spark-connector_2.11/
implementation ("com.exasol:spark-connector_2.11:0.3.2")

Apache Buildr

'com.exasol:spark-connector_2.11:jar:0.3.2'

Apache Ivy

<dependency org="com.exasol" name="spark-connector_2.11" rev="0.3.2">
  <artifact name="spark-connector_2.11" type="jar" />
</dependency>

Groovy Grape

@Grapes(
@Grab(group='com.exasol', module='spark-connector_2.11', version='0.3.2')
)

Scala SBT

libraryDependencies += "com.exasol" % "spark-connector_2.11" % "0.3.2"

Leiningen

[com.exasol/spark-connector_2.11 "0.3.2"]

Dependencies

compile (1)

Group / Artifact	Type	Version
com.exasol » exasol-jdbc	jar	7.0.0

provided (2)

Group / Artifact	Type	Version
org.apache.spark : spark-core_2.11	jar	2.4.5
org.apache.spark : spark-sql_2.11	jar	2.4.5

test (6)

Group / Artifact	Type	Version
org.scalatest : scalatest_2.11	jar	3.2.2
org.scalatestplus : scalatestplus-mockito_2.11	jar	1.0.0-M2
org.mockito : mockito-core	jar	3.5.13
org.testcontainers : jdbc	jar	1.14.3
com.dimafeng : testcontainers-scala_2.11	jar	0.38.4
com.holdenkarau : spark-testing-base_2.11	jar	2.4.5_0.14.0

Project Modules

There are no modules declared in this project.

Spark Exasol Connector

Overview

Spark Exasol Connector supports an integration between Exasol and Apache Spark. It allows creating Spark DataFrame from Exasol queries and saving Spark DataFrame as an Exasol table.

Features

• Creates Spark DataFrame from Exasol query results
• Saves Spark DataFrame to an Exasol table
• Allows configuring the Spark tasks for parallel connection
• Supports Spark DataFrame column pruning and filter push down

Information for Users

• User Guide
• Changelog

Information for Contributors

• Developer Guide

Dependencies

In this section, we define all the dependencies together with their licenses that are required for building, testing and running the connector.

The Java 8 is required for compiling and building the project. In the future versions of Spark, we are planning to change to the newer JVM versions.

Runtime Dependencies

Dependency	Purpose	License
Exasol JDBC	Accessing Exasol using JDBC and sub-connections	MIT License
Spark Core	Apache Spark core libraries for optimized computation	Apache License 2.0
Spark SQL	Apache Spark higher-level SQL and Dataframe interface libraries	Apache License 2.0

Test Dependencies

Dependency	Purpose	License
Scalatest	Testing tool for Scala and Java developers	Apache License 2.0
Scalatest Plus	Integration support between Scalatest and Mockito	Apache License 2.0
Mockito Core	Mocking framework for unit tests	MIT License
Testcontainers JDBC	Testcontainers JDBC to help create JDBC based Docker containers	MIT License
Testcontainers Scala	Scala wrapper for testcontainers-java	MIT License
Spark Testing Base	Library that helps to create tests for Spark applications	Apache License 2.0

Compiler Plugin Dependencies

These plugins help with project development.

Plugin Name	Purpose	License
SBT Coursier	Pure Scala artifact fetching	Apache License 2.0
SBT Wartremover	Flexible Scala code linting tool	Apache License 2.0
SBT Wartremover Contrib	Community managed additional warts for wartremover	Apache License 2.0
SBT Assembly	Create fat jars with all project dependencies	MIT License
SBT API Mappings	Plugin that fetches API mappings for common Scala libraries	Apache License 2.0
SBT Scoverage	Integrates the scoverage code coverage library	Apache License 2.0
SBT Updates	Checks Maven and Ivy repositories for dependency updates	BSD 3-Clause License
SBT Scalafmt	Plugin for https://scalameta.org/scalafmt/ formatting	Apache License 2.0
SBT Scalastyle	Plugin for http://www.scalastyle.org/ Scala style checker	Apache License 2.0
SBT Dependency Graph	Plugin for visualizing dependency graph of your project	Apache License 2.0
SBT Sonatype	Sbt plugin for publishing Scala projects to the Maven central	Apache License 2.0
SBT PGP	PGP plugin for sbt	BSD 3-Clause License
SBT Git	Plugin for Git integration, used to version the release jars	BSD 2-Clause License

Exasol

Exasol AG

Versions

Version
0.3.2 Oct 1, 2020
0.3.1 Aug 31, 2020
0.3.0 Aug 10, 2020
0.2.1 Jul 16, 2019
0.2.0 Jan 21, 2019
0.1.3 Dec 10, 2018
0.1.2 Nov 5, 2018
0.1.1 Oct 25, 2018
0.1.0 Oct 4, 2018