com.indeed:lsmtree-parent

Shared Maven configuration for Indeed.com

License	The Apache Software License, Version 2.0
GroupId	com.indeed
ArtifactId	lsmtree-parent
Last Version	1.0.7
Release Date	May 16, 2017
Type	pom
Description	Shared Maven configuration for Indeed.com
Source Code Management	https://github.com/indeedeng/lsmtree

Download lsmtree-parent

Filename	Size
lsmtree-parent-1.0.7.pom	3 KB

How to add to project

Apache Maven

<!-- https://jarcasting.com/artifacts/com.indeed/lsmtree-parent/ -->
<dependency>
    <groupId>com.indeed</groupId>
    <artifactId>lsmtree-parent</artifactId>
    <version>1.0.7</version>
    <type>pom</type>
</dependency>

Gradle Groovy

// https://jarcasting.com/artifacts/com.indeed/lsmtree-parent/
implementation 'com.indeed:lsmtree-parent:1.0.7'

Gradle Kotlin

// https://jarcasting.com/artifacts/com.indeed/lsmtree-parent/
implementation ("com.indeed:lsmtree-parent:1.0.7")

Apache Buildr

'com.indeed:lsmtree-parent:pom:1.0.7'

Apache Ivy

<dependency org="com.indeed" name="lsmtree-parent" rev="1.0.7">
  <artifact name="lsmtree-parent" type="pom" />
</dependency>

Groovy Grape

@Grapes(
@Grab(group='com.indeed', module='lsmtree-parent', version='1.0.7')
)

Scala SBT

libraryDependencies += "com.indeed" % "lsmtree-parent" % "1.0.7"

Leiningen

[com.indeed/lsmtree-parent "1.0.7"]

Dependencies

There are no dependencies for this project. It is a standalone project that does not depend on any other jars.

Project Modules

• lsmtree-core
• recordcache
• recordlog

Indeed LSM Tree

About

Indeed LSM Tree is a fast key/value store that is efficient for high-volume random access reads and writes.

Indeed uses it for many applications including serving job data for 100s of millions of job searches each day.

Indeed LSM Tree is composed of three parts:

1. lsmtree-core is an implementation of a log-structured merge-tree.
2. recordlog is a library for writing data to append-only record logs optimized for replication.
3. recordcache provides abstractions around writing record logs, building LSM trees, and performing LSM tree lookups.

A typical use case is requiring a key/value store replicated on multiple servers. Instead of replicating the store itself, we store writes in record logs which are slaved to each server. Each server then builds its own LSM tree from the record logs.

Benchmarks

10,000,000 operations using 8 byte keys and 96 byte values

Random writes:

Software	Time
Indeed LSM Tree	272 seconds
Google LevelDB	375 seconds
Kyoto Cabinet B-Tree	375 seconds

Random Reads:

Software	Time
Indeed LSM Tree	46 seconds
Google LevelDB	80 seconds
Kyoto Cabinet B-Tree	183 seconds

Same benchmark using cgroups to limit page cache to 512 MB

Random Reads:

Software	Time
Indeed LSM Tree	454 seconds
Google LevelDB	464 seconds
Kyoto Cabinet B-Tree	50 hours

Indeed Engineering

Versions

Version
1.0.7 May 16, 2017
1.0.6 Mar 30, 2017
1.0.5 Oct 11, 2016