NaviLake

License	MIT
GroupId	tech.navicore
ArtifactId	navilake_2.13
Last Version	1.3.0
Release Date	Jul 10, 2019
Type	jar
Description	NaviLake NaviLake
Project URL	https://github.com/navicore/navilake
Project Organization	tech.navicore
Source Code Management	https://github.com/navicore/navilake

Download navilake_2.13

Filename	Size
navilake_2.13-1.3.0.pom
navilake_2.13-1.3.0.jar	56 KB
navilake_2.13-1.3.0-sources.jar	6 KB
navilake_2.13-1.3.0-javadoc.jar	1 MB

How to add to project

Apache Maven

<!-- https://jarcasting.com/artifacts/tech.navicore/navilake_2.13/ -->
<dependency>
    <groupId>tech.navicore</groupId>
    <artifactId>navilake_2.13</artifactId>
    <version>1.3.0</version>
</dependency>

Gradle Groovy

// https://jarcasting.com/artifacts/tech.navicore/navilake_2.13/
implementation 'tech.navicore:navilake_2.13:1.3.0'

Gradle Kotlin

// https://jarcasting.com/artifacts/tech.navicore/navilake_2.13/
implementation ("tech.navicore:navilake_2.13:1.3.0")

Apache Buildr

'tech.navicore:navilake_2.13:jar:1.3.0'

Apache Ivy

<dependency org="tech.navicore" name="navilake_2.13" rev="1.3.0">
  <artifact name="navilake_2.13" type="jar" />
</dependency>

Groovy Grape

@Grapes(
@Grab(group='tech.navicore', module='navilake_2.13', version='1.3.0')
)

Scala SBT

libraryDependencies += "tech.navicore" % "navilake_2.13" % "1.3.0"

Leiningen

[tech.navicore/navilake_2.13 "1.3.0"]

Dependencies

compile (6)

Group / Artifact	Type	Version
org.scala-lang : scala-library	jar	2.13.0
tech.navicore : navipath_2.13	jar	3.0.2
com.typesafe.scala-logging : scala-logging_2.13	jar	3.9.2
com.microsoft.azure : azure-data-lake-store-sdk	jar	2.3.4
com.typesafe.akka : akka-actor_2.13	jar	2.5.23
com.typesafe.akka : akka-stream_2.13	jar	2.5.23

test (1)

Group / Artifact	Type	Version
org.scalatest : scalatest_2.13	jar	3.0.8

Project Modules

There are no modules declared in this project.

Read Azure Data Lake Storage into Akka Streams

Replay historical data-at-rest into an existing code base that had been designed for streaming.

Current Storage Sources

1. GZip files of UTF8 \n delimited strings
2. Other storage implementations TBD

Uses the adslapi.

USAGE

update your build.sbt dependencies with:

// https://mvnrepository.com/artifact/tech.navicore/navilake
libraryDependencies += "tech.navicore" %% "navilake" % "1.3.0"

This example reads gzip data from Azure Data Lake.

Create a config, a connector, and a source via the example below.

    val consumer = ... // some Sink
    ...
    ...
    ...
    // credentials and location
    implicit val cfg: LakeConfig = LakeConfig(ACCOUNTFQDN, CLIENTID, AUTHEP, CLIENTKEY, Some(PATH))
    val connector: ActorRef = actorSystem.actorOf(GzipConnector.props)
    val src = NaviLake(connector)
    ...
    ...
    ...
    src.runWith(consumer)
    ...
    ...
    ...

---

Versions

Version
1.3.0 Jul 10, 2019