angel-math

a stand alone machine learning suite which can easy to integrate with angel ps

License

License

GroupId

GroupId

com.tencent.angel
ArtifactId

ArtifactId

angel-math
Last Version

Last Version

0.1.1
Release Date

Release Date

Type

Type

jar
Description

Description

angel-math
a stand alone machine learning suite which can easy to integrate with angel ps
Project URL

Project URL

https://github.com/Angel-ML/math2
Source Code Management

Source Code Management

https://github.com/Angel-ML/math2

Download angel-math

How to add to project

<!-- https://jarcasting.com/artifacts/com.tencent.angel/angel-math/ -->
<dependency>
    <groupId>com.tencent.angel</groupId>
    <artifactId>angel-math</artifactId>
    <version>0.1.1</version>
</dependency>
// https://jarcasting.com/artifacts/com.tencent.angel/angel-math/
implementation 'com.tencent.angel:angel-math:0.1.1'
// https://jarcasting.com/artifacts/com.tencent.angel/angel-math/
implementation ("com.tencent.angel:angel-math:0.1.1")
'com.tencent.angel:angel-math:jar:0.1.1'
<dependency org="com.tencent.angel" name="angel-math" rev="0.1.1">
  <artifact name="angel-math" type="jar" />
</dependency>
@Grapes(
@Grab(group='com.tencent.angel', module='angel-math', version='0.1.1')
)
libraryDependencies += "com.tencent.angel" % "angel-math" % "0.1.1"
[com.tencent.angel/angel-math "0.1.1"]

Dependencies

compile (8)

Group / Artifact Type Version
org.scala-lang : scala-library jar 2.11.8
it.unimi.dsi : fastutil jar 7.1.0
org.apache.velocity : velocity jar 1.7
com.github.fommil.netlib : all pom 1.1.2
commons-logging : commons-logging jar 1.1.1
org.slf4j : slf4j-log4j12 jar 1.6.2
org.slf4j : slf4j-api jar 1.6.2
com.google.guava : guava jar 12.0

test (2)

Group / Artifact Type Version
org.scalatest : scalatest_2.11 jar 2.2.6
org.scalanlp : breeze_2.11 jar 0.13.1

Project Modules

There are no modules declared in this project.

math2

A math lib optimized for sparse calculation.

<dependency>
    <groupId>com.tencent.angel</groupId>
    <artifactId>angel-math</artifactId>
    <version>0.1.1</version>
</dependency>

The math lib is a self-developed new subject separated from previous Angel version. Other projects such as mlcore, parameter server and Angel serving are based math lib. The math lib is designed for sparse calculation. The main features of math lib as following:

1. Smart rehash

We adopt a fast typed hash map (no general type) to store sparse vectors and matrix. As the calculation going, the vectors is getting more and more dense. As a result, many rehash emerged. To reduce the amount of rehash, we specify an operation type to every operator.

  • the union type represents the output index is the union of the input indices;
  • the intersection type indicates the output index is the intersection of the input indices;
  • the all type denotes the result would be dense.

Once we known the operator type, we can use it reduce rehash greatly.

operator type example rehash
union $x + y$ choose out = coeff * (|indices x|+|indices of y|) , at most once rehash
intersection $x * y$ choose out = min(|indices x|, |indices of y|), no rehash
all $x - 1$ to dense, no rehash

For example, if we know the operator type is intersection, the result size is no larger than the smaller input, so we can pre-allocate space to prevent rehash.

2. Storage aware (Storage siwtch)

As mentioned in Smart rehash, the vectors is getting denser as the calculation going. If the sparsity exceeds a certain threshold, we choose to switch the sparse storage to dense one to promote the calculation efficiency.

figure1

There is a relationship between Storage aware and Smart rehash. figure2 If the sparsity the below some threshold, we choose Smart rehash, and over the threshold, choose Storage siwtch.

3. Executor-expression mechanism

Expression folding is an excellent feature for reducing function calls and iteration. Unfortunately, there is no expression folding in our math lib. However, we provide an executor-expression mechanism to allow use do expression folding manually.

for example, we can fold the expression in the red box manually to reduce calculation. figure3

At last, User friendly. User can create vector and matrix for factory classes VFactory and MFactory respectively. After the vector and matrix created, they can be used as if you are doing mathematical deduction without worrying about the data type and storage.

com.tencent.angel

Angel

Angel is a high-performance and full-stack distributed ML platform. It is an LF AI Graduation project.

Versions

Version
0.1.1
0.1.0