sparkling-water-core

Sparkling Water Core

License	License The Apache Software License, Version 2.0
Categories	Categories H2 Data Databases H2O Business Logic Libraries Machine Learning
GroupId	GroupId ai.h2o
ArtifactId	ArtifactId sparkling-water-core_2.10
Last Version	Last Version 1.6.13
Release Date	Release Date Sep 12, 2017
Type	Type jar
Description	Description sparkling-water-core Sparkling Water Core
Project URL	Project URL https://github.com/h2oai/sparkling-water
Project Organization	Project Organization H2O.ai
Source Code Management	Source Code Management https://github.com/h2oai/sparkling-water

Download sparkling-water-core_2.10

Filename	Size
sparkling-water-core_2.10-1.6.13.pom
sparkling-water-core_2.10-1.6.13.jar	161 KB
sparkling-water-core_2.10-1.6.13-sources.jar	22 KB
sparkling-water-core_2.10-1.6.13-scaladoc.jar	386 KB
sparkling-water-core_2.10-1.6.13-javadoc.jar	261 bytes
Browse

How to add to project

Apache Maven

<!-- https://jarcasting.com/artifacts/ai.h2o/sparkling-water-core_2.10/ -->
<dependency>
    <groupId>ai.h2o</groupId>
    <artifactId>sparkling-water-core_2.10</artifactId>
    <version>1.6.13</version>
</dependency>

Gradle Groovy

// https://jarcasting.com/artifacts/ai.h2o/sparkling-water-core_2.10/
implementation 'ai.h2o:sparkling-water-core_2.10:1.6.13'

Gradle Kotlin

// https://jarcasting.com/artifacts/ai.h2o/sparkling-water-core_2.10/
implementation ("ai.h2o:sparkling-water-core_2.10:1.6.13")

Apache Buildr

'ai.h2o:sparkling-water-core_2.10:jar:1.6.13'

Apache Ivy

<dependency org="ai.h2o" name="sparkling-water-core_2.10" rev="1.6.13">
  <artifact name="sparkling-water-core_2.10" type="jar" />
</dependency>

Groovy Grape

@Grapes(
@Grab(group='ai.h2o', module='sparkling-water-core_2.10', version='1.6.13')
)

Scala SBT

libraryDependencies += "ai.h2o" % "sparkling-water-core_2.10" % "1.6.13"

Leiningen

[ai.h2o/sparkling-water-core_2.10 "1.6.13"]

Dependencies

compile (19)

Group / Artifact	Type	Version
ai.h2o : h2o-genmodel	jar	3.14.0.2
ai.h2o : h2o-core	jar	3.14.0.2
ai.h2o : h2o-algos	jar	3.14.0.2
ai.h2o : h2o-web	jar	3.14.0.2
ai.h2o : h2o-avro-parser	jar	3.14.0.2
ai.h2o : h2o-ext-xgboost	jar	3.14.0.2
ai.h2o : h2o-genmodel-ext-xgboost	jar	3.14.0.2
ai.h2o : h2o-parquet-parser	jar	3.14.0.2
ai.h2o : h2o-orc-parser	jar	3.14.0.2
ai.h2o : h2o-scala_2.10	jar	3.14.0.2
ai.h2o : h2o-persist-hdfs	jar	3.14.0.2
ai.h2o : h2o-persist-s3	jar	3.14.0.2
ai.h2o : h2o-automl	jar	3.14.0.2
org.apache.spark : spark-core_2.10	jar	1.6.3
org.apache.spark : spark-sql_2.10	jar	1.6.3
org.apache.spark : spark-mllib_2.10	jar	1.6.3
ai.h2o : sparkling-water-repl_2.10	jar	1.6.13
org.joda : joda-convert	jar	1.7
org.scala-lang : scala-library	jar	2.10.6

test (2)

Group / Artifact	Type	Version
org.scalatest : scalatest_2.10	jar	2.2.1
junit : junit	jar	4.11

Project Modules

There are no modules declared in this project.

Sparkling Water

Sparkling Water integrates H₂O's fast scalable machine learning engine with Spark. It provides:

Utilities to publish Spark data structures (RDDs, DataFrames, Datasets) as H2O's frames and vice versa.
DSL to use Spark data structures as input for H2O's algorithms.
Basic building blocks to create ML applications utilizing Spark and H2O APIs.
Python interface enabling use of Sparkling Water directly from PySpark.

Getting Started

User Documentation

The documentation contains also documentation for our clients, PySparkling and RSparkling.

Download Binaries

Maven

Each Sparkling Water release is published into Maven central with following coordinates:

ai.h2o:sparkling-water-core_{{scala_version}}:{{version}} - Includes core of Sparkling Water
ai.h2o:sparkling-water-examples_{{scala_version}}:{{version}} - Includes example applications
ai.h2o:sparkling-water-repl_{{scala_version}}:{{version}} - Spark REPL integration into H2O Flow UI
ai.h2o:sparkling-water-ml_{{scala_version}}:{{version}} - Extends Spark ML package by H2O-based transformations
ai.h2o:sparkling-water-scoring_{{scala_version}}:{{version}} - Lightweight package used for Scoring with H2O MOJOs. This package does not have run-time dependency on H2O cluster.
ai.h2o:sparkling-water-package_{{scala_version}}:{{version}} - Uber Sparkling Water package containing all dependencies. This is designed to use as Spark package via --packages option

Note: The {{version}} references to a release version of Sparkling Water, the {{scala_version}} references to Scala base version.

The full list of published packages is available here.

Sparkling Water Requirements for Spark 3.0

Linux/OS X/Windows
Java 8+
Python 2.7+ For Python version of Sparkling Water (PySparkling)
Spark 3.0 and SPARK_HOME shell variable must point to your local Spark installation

To see requirements for older Spark version, please visit relevant documentation.

Use Sparkling Water

Sparkling Water is distributed as a Spark application library which can be used by any Spark application. Furthermore, we provide also zip distribution which bundles the library and shell scripts.

There are several ways of using Sparkling Water:

Sparkling Shell
Sparkling Water driver
Spark Shell and include Sparkling Water library via --jars or --packages option
Spark Submit and include Sparkling Water library via --jars or --packages option
PySpark with PySparkling

Run Sparkling shell

The Sparkling shell encapsulates a regular Spark shell and append Sparkling Water library on the classpath via --jars option. The Sparkling Shell supports creation of an H₂O cloud and execution of H₂O algorithms.

Either download or build Sparkling Water
Configure the location of Spark cluster:
```
export SPARK_HOME="/path/to/spark/installation"
export MASTER="local[*]"
```
In this case, local[*] points to an embedded single node cluster.
Run Sparkling Shell:
```
bin/sparkling-shell
```
Sparkling Shell accepts common Spark Shell arguments. For example, to increase memory allocated by each executor, use the spark.executor.memory parameter: bin/sparkling-shell --conf "spark.executor.memory=4g"
Initialize H2OContext
```
import ai.h2o.sparkling._
val hc = H2OContext.getOrCreate()
```
H2OContext starts H2O services on top of Spark cluster and provides primitives for transformations between H₂O and Spark data structures.

Use Sparkling Water with PySpark

Sparkling Water can be also used directly from PySpark and the integration is called PySparkling.

See PySparkling README to learn about PySparkling.

Use Sparkling Water via Spark Packages

To see how Sparkling Water can be used as Spark package, please see Use as Spark Package.

Use Sparkling Water in Windows environments

See Windows Tutorial to learn how to use Sparkling Water in Windows environments.

Sparkling Water examples

To see how to run examples for Sparkling Water, please see Running Examples.

Sparkling Water Backends

Sparkling water supports two backend/deployment modes - internal and external. Sparkling Water applications are independent on the selected backend. The backend can be specified before creation of the H2OContext.

For more details regarding the internal or external backend, please see Backends.

FAQ

List of all Frequently Asked Questions is available at FAQ.

Development

Complete development documentation is available at Development Documentation.

Build Sparkling Water

To see how to build Sparkling Water, please see Build Sparkling Water.

Develop applications with Sparkling Water

An application using Sparkling Water is regular Spark application which bundling Sparkling Water library. See Sparkling Water Droplet providing an example application here.

Contributing

Look at our list of JIRA tasks or send your idea to [email protected].

Filing Bug Reports and Feature Requests

You can file a bug report of feature request directly in the Sparkling Water JIRA page at http://jira.h2o.ai/.

Log in to the Sparkling Water JIRA tracking system. (Create an account if necessary.)
Once inside the home page, click the Create button.
A form will display allowing you to enter information about the bug or feature request.

Enter the following on the form:
- Select the Project that you want to file the issue under. For example, if this is an open source public bug, you should file it under SW (SW).
- Specify the Issue Type. For example, if you believe you've found a bug, then select Bug, or if you want to request a new feature, then select New Feature.
- Provide a short but concise summary about the issue. The summary will be shown when engineers organize, filter, and search for Jira tickets.
- Specify the urgency of the issue using the Priority dropdown menu.
- If there is a due date specify it with the Due Date.
- The Components drop down refers to the API or language that the issue relates to. (See the drop down menu for available options.)
- You can leave Affects Version/s, Fix Versions, and Assignee fields blank. Our engineering team will fill this in.
- Add a detailed description of your bug in the Description section. Best practice for descriptions include:
- A summary of what the issue is
- What you think is causing the issue
- Reproducible code that can be run end to end without requiring an engineer to edit your code. Use {code} {code} around your code to make it appear in code format.
- Any scripts or necessary documents. Add by dragging and dropping your files into the create issue dialogue box.
You can be able to leave the rest of the ticket blank.
When you are done with your ticket, simply click on the Create button at the bottom of the page.

After you click Create, a pop up will appear on the right side of your screen with a link to your Jira ticket. It will have the form https://0xdata.atlassian.net/browse/SW-####. You can use this link to later edit your ticket.

Please note that your Jira ticket number along with its summary will appear in one of the Jira ticket slack channels, and anytime you update the ticket anyone associated with that ticket, whether as the assignee or a watcher will receive an email with your changes.

Have Questions?

We also respond to questions tagged with sparkling-water and h2o tags on the Stack Overflow.

Change Logs

Change logs are available at Change Logs.

H2O.ai

Fast Scalable Machine Learning For Smarter Applications

Versions

Version
1.6.13 Sep 12, 2017
1.6.12 Jun 29, 2017
1.6.11 May 25, 2017
1.6.10 Apr 21, 2017
1.6.9 Apr 4, 2017
1.6.8 Sep 26, 2016
1.6.7 Sep 2, 2016
1.6.6 Aug 31, 2016
1.6.5 Jun 7, 2016
1.6.4 Jun 7, 2016
1.6.3 Apr 28, 2016
1.6.2 Apr 26, 2016
1.6.1 Mar 16, 2016
1.5.16 Jul 8, 2016
1.5.15 Jul 7, 2016
1.5.14 Apr 28, 2016
1.5.13 Apr 26, 2016
1.5.12 Mar 11, 2016
1.5.11 Mar 8, 2016
1.5.10 Jan 13, 2016
1.5.9 Jan 8, 2016
1.5.6 Nov 7, 2015
1.5.3 Oct 20, 2015
1.5.2 Sep 29, 2015
1.5.1 Sep 28, 2015
1.4.15 Jul 7, 2016
1.4.14 Apr 28, 2016
1.4.13 Apr 26, 2016
1.4.12 Mar 18, 2016
1.4.11 Mar 18, 2016
1.4.9 Jan 8, 2016
1.4.8 Dec 2, 2015
1.4.7 Oct 21, 2015
1.4.6 Sep 29, 2015
1.4.5 Sep 29, 2015
1.4.4 Sep 23, 2015
1.4.3 Jul 6, 2015
1.3.10 Sep 29, 2015
1.3.9 Sep 29, 2015
1.3.8 Sep 23, 2015
1.3.7 Jul 6, 2015
1.3.6 Jun 27, 2015
1.3.5 Jun 15, 2015
1.3.4 Jun 12, 2015
1.3.3 Jun 5, 2015
1.3.2 May 27, 2015
1.3.1 May 26, 2015
1.2.6 May 26, 2015
1.2.5 May 20, 2015
1.2.3 May 20, 2015
0.2.14 May 14, 2015
0.2.13 May 2, 2015
0.2.12 Apr 22, 2015
0.2.11 Feb 28, 2015
0.2.10 Feb 14, 2015
0.2.9 Feb 5, 2015
0.2.8 Feb 4, 2015
0.2.7 Feb 4, 2015
0.2.6 Jan 31, 2015
0.2.5 Jan 28, 2015
0.2.3 Jan 21, 2015
0.2.2 Jan 20, 2015
0.2.1 Jan 11, 2015
0.2.0 Oct 24, 2014