quasar-foundation-internal


License

License

Categories

Categories

Quasar Application Layer Libs Distributed Applications
GroupId

GroupId

org.quasar-analytics
ArtifactId

ArtifactId

quasar-foundation-internal_2.12
Last Version

Last Version

34.2.5
Release Date

Release Date

Type

Type

jar
Description

Description

quasar-foundation-internal
quasar-foundation-internal
Project URL

Project URL

https://github.com/quasar-analytics/quasar
Project Organization

Project Organization

SlamData Inc.
Source Code Management

Source Code Management

https://github.com/quasar-analytics/quasar

Download quasar-foundation-internal_2.12

How to add to project

<!-- https://jarcasting.com/artifacts/org.quasar-analytics/quasar-foundation-internal_2.12/ -->
<dependency>
    <groupId>org.quasar-analytics</groupId>
    <artifactId>quasar-foundation-internal_2.12</artifactId>
    <version>34.2.5</version>
</dependency>
// https://jarcasting.com/artifacts/org.quasar-analytics/quasar-foundation-internal_2.12/
implementation 'org.quasar-analytics:quasar-foundation-internal_2.12:34.2.5'
// https://jarcasting.com/artifacts/org.quasar-analytics/quasar-foundation-internal_2.12/
implementation ("org.quasar-analytics:quasar-foundation-internal_2.12:34.2.5")
'org.quasar-analytics:quasar-foundation-internal_2.12:jar:34.2.5'
<dependency org="org.quasar-analytics" name="quasar-foundation-internal_2.12" rev="34.2.5">
  <artifact name="quasar-foundation-internal_2.12" type="jar" />
</dependency>
@Grapes(
@Grab(group='org.quasar-analytics', module='quasar-foundation-internal_2.12', version='34.2.5')
)
libraryDependencies += "org.quasar-analytics" % "quasar-foundation-internal_2.12" % "34.2.5"
[org.quasar-analytics/quasar-foundation-internal_2.12 "34.2.5"]

Dependencies

compile (17)

Group / Artifact Type Version
org.scala-lang : scala-library jar 2.12.4
com.slamdata : slamdata-predef_2.12 jar 0.0.4
org.scalaz : scalaz-core_2.12 jar 7.2.18
org.scalaz : scalaz-concurrent_2.12 jar 7.2.18
org.scalaz.stream : scalaz-stream_2.12 jar 0.8.6a
com.github.julien-truffaut : monocle-core_2.12 jar 1.4.0
org.typelevel : algebra_2.12 jar 0.7.0
org.typelevel : spire_2.12 jar 0.14.1
io.argonaut : argonaut_2.12 jar 6.2
io.argonaut : argonaut-scalaz_2.12 jar 6.2
com.slamdata : matryoshka-core_2.12 jar 0.18.3
com.slamdata : pathy-core_2.12 jar 0.2.11
com.slamdata : pathy-argonaut_2.12 jar 0.2.11
eu.timepit : refined_2.12 jar 0.8.3
com.chuusai : shapeless_2.12 jar 2.3.2
org.scalacheck : scalacheck_2.12 jar 1.13.4
com.propensive : contextual_2.12 jar 1.0.1

test (9)

Group / Artifact Type Version
com.github.mpilquist : simulacrum_2.12 jar 0.10.0
org.typelevel : algebra-laws_2.12 jar 0.7.0
org.typelevel : discipline_2.12 jar 0.7.2
org.typelevel : spire-laws_2.12 jar 0.14.1
org.specs2 : specs2-core_2.12 jar 4.0.2
org.specs2 : specs2-scalacheck_2.12 jar 4.0.2
org.scalaz : scalaz-scalacheck-binding_2.12 jar 7.2.18-scalacheck-1.13
org.typelevel : shapeless-scalacheck_2.12 jar 0.6.1
org.typelevel : scalaz-specs2_2.12 jar 0.5.2

Project Modules

There are no modules declared in this project.

Quasar Discord

Quasar is a purely-functional compiler and optimizing planner for queries expressed in terms of the Multidimensional Relational Algebra (MRA). Quasar has support for arbitrary backends, both heavyweight (full evaluation engines) and lightweight (simple reads with optional pushdown of structural operations and columnar predicates), including full classpath isolation for lightweight backends.

It's important to note that Quasar is not, in and of itself, a runnable application. It is a library which is used by the broader Precog product, much of which is closed-source. Contributions are very much welcome, as is feedback, questions, and general conversation. Join the Discord!

Building and Testing

Quasar builds with SBT:

$ ./sbt
> test:compile
> test

If running on Windows, you may use the SBT batch file instead of the shell script.

Code Organization

Probably the most interesting part of the codebase is the optimizing query planner, which is implemented in the qsu submodule, based on data structures defined in qscript. I recommend starting by looking at the LPtoQS class, which defines a kleisli composition that clearly lays out all of the phases of the compiler. The core data structure used by the compiler is QSUGraph, which is a purely functional representation of a directed acyclic graph, which in turn represents data flow in a query.

The formulation of the query plan itself is a fixed-point data structure dictated by several pattern functors composed via coproducts. The primary such pattern functor is QScriptCore. You would generally deconstruct and interpret this query plan using general folds provided by matryoshka.

Query operations which are pushed down to the underlying data source are represented by ScalarStage, and carried via InterpretedRead. Data sources are always free to only implement a subset of the pushdown functionality.

The codebase makes extremely heavy use of Scalaz and Cats throughout (using shims to solve the impedance between them), and many high-level operations (such as datasets) are represented as fs2 streams.

Local Datasource

A Datasource implementation providing access to the filesystems local to the JVM.

Configuration for the local datasource has the following JSON format

{
  "rootDir": String,
  "format": {
    "type": "json",
    "variant": "array-wrapped" | "line-delimited",
    [precise: Boolean]
  },
  ["readChunkSizeBytes": Number,]
  ["compressionScheme": "gzip"]
}
  • rootDir an absolute path to a local directory at which to root the datasource, all paths handled by the datasource will be interpreted relative to this physical directory.
  • format the format of all resources in the datasource, currently JSON is supported in both array-wrapped and line-delimited variants.
  • readChunkSizeBytes (optional) an integer indicating the chunk size to use when reading local files, the default is 1048576 (1MB). Different values may yield higher throughput depending on the filesystem.
  • compressionScheme (optional) whether to expect resources to be compressed, currently gzip is the only supported compression scheme. Omitting this option indicates uncompressed resources.

Legal

Copyright © 2020 Precog Data

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

org.quasar-analytics

Quasar

An Apache 2-licensed analytics engine for semi-structured data.

Versions

Version
34.2.5
34.2.4
34.2.3
34.0.1
34.0.0
33.0.0
32.1.3
32.0.2
32.0.1
31.0.0
30.1.27
30.1.25
30.1.24
30.1.23
30.1.22
30.1.21
30.1.20
30.1.16
30.1.14
30.1.13
30.1.12
30.1.11
30.1.9
30.1.8
30.1.7
30.1.5
30.1.3
30.1.2
30.1.1
30.0.6
30.0.3
30.0.2
30.0.1
30.0.0
29.1.18
29.1.17
29.1.16
29.1.15
29.1.14
29.1.12
29.1.9
29.1.6
29.1.5
29.1.4
29.1.3