Compress-LZF

Compression codec for LZF encoding for particularly encoding/decoding, with reasonable compression. Compressor is basic Lempel-Ziv codec, without Huffman (deflate/gzip) or statistical post-encoding. See "http://oldhome.schmorp.de/marc/liblzf.html" for more on original LZF package.

License

License

GroupId

GroupId

com.ning
ArtifactId

ArtifactId

compress-lzf
Last Version

Last Version

1.1
Release Date

Release Date

Type

Type

bundle
Description

Description

Compress-LZF
Compression codec for LZF encoding for particularly encoding/decoding, with reasonable compression. Compressor is basic Lempel-Ziv codec, without Huffman (deflate/gzip) or statistical post-encoding. See "http://oldhome.schmorp.de/marc/liblzf.html" for more on original LZF package.
Project URL

Project URL

https://github.com/ning/compress
Project Organization

Project Organization

FasterXML
Source Code Management

Source Code Management

https://github.com/ning/compress

Download compress-lzf

Dependencies

test (1)

Group / Artifact Type Version
org.testng : testng jar 6.8.21

Project Modules

There are no modules declared in this project.

LZF Compressor

Overview

LZF-compress is a Java library for encoding and decoding data in LZF format, written by Tatu Saloranta ([email protected])

Data format and algorithm based on original LZF library by Marc A Lehmann. See LZF Format Specification for full description.

Format differs slightly from some other adaptations, such as the one used by H2 database project (by Thomas Mueller); although internal block compression structure is the same, block identifiers differ. This package uses the original LZF identifiers to be 100% compatible with existing command-line lzf tool(s).

LZF algorithm itself is optimized for speed, with somewhat more modest compression. Compared to the standard Deflate (algorithm gzip uses) LZF can be 5-6 times as fast to compress, and twice as fast to decompress. Compression rate is lower since no Huffman-encoding is used after lempel-ziv substring elimination.

License

Apache License 2.0

Requirements

Version up to 1.0.4 require JDK 6; version 1.1 requires JDK 8.

Library has no external dependencies.

Usage

See Wiki for more details; here's a "TL;DNR" version.

Both compression and decompression can be done either by streaming approach:

InputStream in = new LZFInputStream(new FileInputStream("data.lzf"));
OutputStream out = new LZFOutputStream(new FileOutputStream("results.lzf"));
InputStream compIn = new LZFCompressingInputStream(new FileInputStream("stuff.txt"));

or by block operation:

byte[] compressed = LZFEncoder.encode(uncompressedData);
byte[] uncompressed = LZFDecoder.decode(compressedData);

and you can even use the LZF jar as a command-line tool (it has manifest that points to 'com.ning.compress.lzf.LZF' as the class having main() method to call), like so:

java -jar compress-lzf-1.0.3.jar

(which will display necessary usage arguments for -c(ompressing) or -d(ecompressing) files.

Adding as Dependency

Maven

<dependency>
  <groupId>com.ning</groupId>
  <artifactId>compress-lzf</artifactId>
  <version>1.1</version>
</dependency>

Module info (JPMS)

Starting with version 1.1, module-info.class is included; module name is com.ning.compress.lzf so you will need to use:

requires com.ning.compress.lzf

Parallel processing

Since the compression is more CPU-heavy than decompression, it could benefit from concurrent operation. This works well with LZF because of its block-oriented nature, so that although there is need for sequential processing within block (of up to 64kB), encoding of separate blocks can be done completely independently: there are no dependencies to earlier blocks.

The main abstraction to use is PLZFOutputStream which a FilterOutputStream and implements java.nio.channels.WritableByteChannel as well. It use is like that of any OutputStream:

PLZFOutputStream output = new PLZFOutputStream(new FileOutputStream("stuff.lzf"));
// then write contents:
output.write(buffer);
// ...
output.close();

Interoperability

Besides Java support, LZF codecs / bindings exist for non-JVM languages as well:

Related

Check out jvm-compress-benchmark for comparison of space- and time-efficiency of this LZF implementation, relative other available Java-accessible compression libraries.

More

Project Wiki.

Alternative High-Speed Lempel-Ziv Compressors

LZF belongs to a family of compression codecs called "simple Lempel-Ziv" codecs. Since LZ compression is also the first part of deflate compression (which is used, along with simple framing, for gzip), it can be viewed as "first-part of gzip" (second part being Huffman-encoding of compressed content).

There are many other codecs in this category, most notable (and competitive being)

all of which have very similar compression ratios (due to same underlying algorithm, differences coming from slight encoding variations, and efficiency differences in back-reference matching), and similar performance profiles regarding ratio of compression vs uncompression speeds.

com.ning

Ning

Versions

Version
1.1
1.0.4
1.0.3
1.0.2
1.0.1
1.0.0
0.9.10
0.9.9
0.9.8
0.9.7
0.9.6
0.9.5
0.9.4
0.9.3
0.9.2
0.9.1
0.9.0
0.8.6
0.8.5
0.8.4
0.8.3
0.8.2
0.8.1
0.8.0
0.7.0
0.6.0
0.5.3