zstd-codec

WebJar for zstd-codec

License	MIT
GroupId	org.webjars.npm
ArtifactId	zstd-codec
Last Version	0.1.1
Release Date	Dec 29, 2018
Type	jar
Description	zstd-codec WebJar for zstd-codec
Project URL	http://webjars.org
Source Code Management	https://github.com/yoshihitoh/zstd-codec

Download zstd-codec

Filename	Size
zstd-codec-0.1.1.pom
zstd-codec-0.1.1.jar	3 MB
zstd-codec-0.1.1-sources.jar	22 bytes
zstd-codec-0.1.1-javadoc.jar	22 bytes

How to add to project

Apache Maven

<!-- https://jarcasting.com/artifacts/org.webjars.npm/zstd-codec/ -->
<dependency>
    <groupId>org.webjars.npm</groupId>
    <artifactId>zstd-codec</artifactId>
    <version>0.1.1</version>
</dependency>

Gradle Groovy

// https://jarcasting.com/artifacts/org.webjars.npm/zstd-codec/
implementation 'org.webjars.npm:zstd-codec:0.1.1'

Gradle Kotlin

// https://jarcasting.com/artifacts/org.webjars.npm/zstd-codec/
implementation ("org.webjars.npm:zstd-codec:0.1.1")

Apache Buildr

'org.webjars.npm:zstd-codec:jar:0.1.1'

Apache Ivy

<dependency org="org.webjars.npm" name="zstd-codec" rev="0.1.1">
  <artifact name="zstd-codec" type="jar" />
</dependency>

Groovy Grape

@Grapes(
@Grab(group='org.webjars.npm', module='zstd-codec', version='0.1.1')
)

Scala SBT

libraryDependencies += "org.webjars.npm" % "zstd-codec" % "0.1.1"

Leiningen

[org.webjars.npm/zstd-codec "0.1.1"]

Dependencies

There are no dependencies for this project. It is a standalone project that does not depend on any other jars.

Project Modules

There are no modules declared in this project.

zstd-codec

Zstandard codec for Node.js and Web, powered by Emscripten.

Languages

• English

Description

zstd-codec is a binding of Zstandard for Node.js and Browsers, includes JavaScript port of Zstandard compiled with Emscripten.

Installation

npm

npm install zstd-codec

yarn

yarn add zstd-codec

Usage

require module, and instantiate api objects.

const ZstdCodec = require('zstd-codec').ZstdCodec;
ZstdCodec.run(zstd => {
    const simple = new zstd.Simple();
    const streaming = new zstd.Streaming();
});

• Use Simple API for small data
• Use Streaming API for large data

Simple API

• Using Zstandard's Simple API
  • ZSTD_compress for compress
  • ZSTD_decompress for decompress
• Store whole input/output bytes into Emscripten's heap
  • Available Emscripten's heap size is 16MiB
  • (input.length + output.length) should be less than 12MiB

compress(content_bytes, compression_level)

• content_bytes: data to compress, must be Uint8Array.
• compression_level: (optional) compression level, default value is 3

// prepare data to compress
const data = ...;

// compress
const level = 5;
const compressed = simple.compress(data, level);

// handle compressed data
do_something(compressed);

decompress(compressed_bytes)

• compressed_bytes: data to decompress, must be Uint8Array.

// prepare compressed data
const compressed = ...;

// decompress
const data = simple.decompress(compressed);

// handle decompressed data
do_something(data);

Streaming APIs

• Using Zstandard's Streaming API
  • ZSTD_xxxxCStream APIs for compress
  • ZSTD_xxxxDStream APIs for decompress
• Store partial input/output bytes into Emscripten's heap

const streaming = new ZstdCodec.Streaming();

You can use custom Iterable object on compressChunks / decompressChunks.

compress(content_bytes, compression_level)

• content_bytes: data to compress, must be 'Uint8Array'
• compression_level: (optional) compression level, default value is 3

const compressed = streaming.compress(data); // use default compression_level 3

compressChunks(chunks, size_hint, compression_level)

• chunks: data chunks to compress, must be Iterable of Uint8Array
• size_hint: (optional) size hint to store compressed data (to improve performance)
• compression_level: (optional) compression level, default value is 3

const chunks = [dataPart1, dataPart2, dataPart3, ...];
const size_hint = chunks.map((ar) => ar.length).reduce((p, c) => p + c);
const compressed = streaming.compressChunks(chunks, size_hint); // use default compression_level 3

decompress(compressed_bytes, size_hint)

• compressed_bytes: data to decompress, must be Uint8Array.
• size_hint: (optional) size hint to store decompressed data (to improve performance)

const data = streaming.decompress(data); // can omit size_hint

decompressChunks(chunks, size_hint)

• chunks: data chunks to compress, must be Iterable of Uint8Array
• size_hint: (optional) size hint to store compressed data (to improve performance)

const chunks = [dataPart1, dataPart2, dataPart3, ...];
const size_hint = 2 * 1024 * 1024; // 2MiB
const data = streaming.decompressChunks(chunks, size_hint);

Dictionary API

const ZstdCodec = require('zstd-codec').ZstdCodec;
ZstdCodec.run(zstd => {
    const simple = new zstd.Simple();

    // compress using trained dictionary
    const cdict = new zstd.Dict.Compression(dict_bytes, compression_level);
    const compressed = simple.compressUsingDict(data, cdict);

    // decompress using trained dictionary
    const ddict = new zstd.Dict.Decompression(dict_bytes);
    const data = simple.decompressUsingDict(compressed, ddict);
});

Migrate from v0.0.x to v0.1.x

API changed

please use callback style module instantiation.

// v0.0.x
const zstd = require('zstd-codec').ZstdCodec;
const simple = new zstd.Simple();

// v0.1.x
const ZstdCodec = require('zstd-codec').ZstdCodec;
ZstdCodec.run(zstd => {
    const simple = new zstd.Simple();
});

NOTE: I wanted to use Promise instead of callback, but does not work :( Need to survey why promise does not work, but it will take a lot of times.

Class name changed

• ZstdCompressionDict => zsdt.Dict.Compression
• ZstdDecompressionDict => zsdt.Dict.Decompression

Example

Browser

See the document.

Node.js

TODO: add an example for Node.js.

TODO

• add CI (Travis CI or Circle CI?)
• improve APIs
• write this document
• add how to build zstd with Emsxcripten
• add how to test
• performance test
• add more tests

Versions

Version
0.1.1 Dec 29, 2018