JarCastingMetrics Library for Java
An application metrics client integrated with Graphite/OpenTSDB/more to come. Offers the following features:
- sends the metrics in batch to the backend
- fail safe (retry + circuit breaker to not overwhelm the backend)
- send to multiple backends in parallel
- useful for migrating to one backend to another
- useful for sending a set of metrics to one backend (e.g. a sub-set of importants metrics used for alerts in Prometheus) and another set to another backend (e.g. all metrics to OpenTSDB).
Usage
Maven
<!-- https://mvnrepository.com/artifact/com.adobe.aam/metrics-all -->
<dependency>
<groupId>com.adobe.aam</groupId>
<artifactId>metrics-all</artifactId>
<version>1.0.2</version>
</dependency>
Gradle
// https://mvnrepository.com/artifact/com.adobe.aam/metrics-all
compile group: 'com.adobe.aam', name: 'metrics-all', version: '1.0.2'
Sample application
You can find a demo app in this project: Sample Application
Java docs
You can find here the Java docs
Architecture
Sample App: How to use the metrics library
See metrics-sample app, for a fully fledged demo.
Create a metric
There are several metric types available: CounterMetric, AverageMetric, MaxMetric, MinMetric etc.
Average Metric
// Create a metric that generates the average of the values.
Metric metric = Metric.newInstance("request.time", Metric.Type.AVG);
metric.track(10);
metric.track(20);
metric.get(); // returns (10 + 20) / 2 = 15
Counter
CounterMetric metric = new CounterMetric("event.pixel");
metric.increment(); // New value is 1
metric.increment(); // New value is 2
metric.add(10); // New value is 12
metric.get(); // returns 12
Send a metric to a backend (e.g. Graphite / OpenTSDB)
How the metric client manages to publish metrics to the backend
By default, the client is able to send metrics to Backend using a retry mechanism guarded by a circuit breaker mechanism. The former ensures metrics are being resent should the initial request fail, providing a way of not losing important metrics. On the other hand, the circuit breaker mechanism makes the client silently aware of a non-responsive Backend backend by discarding all incoming metrics until the service is up and running again.
Nevertheless, one can always switch to a client w/o retry and/or w/o circuit breaker mechanisms depending on the environment needs. For instance, for a testing environment that does not highly rely on metrics, the metric client can be used without retry and circuit breaker mechanisms.
However, there is a great benefit brought by the enablement of both mechanism (via config) - safely and effectively (using retry) sending metrics while not polluting with metrics the memory of the metric client when the Backend is non-responsive. The downside of enabling the circuit breaker is the potential impact on aggregated metrics - each client node might enter into an open circuit at a different time, hence the metrics are prone to inconsistency
References:
Create a metric client
The MetricClientFactory contains a series of methods to create a metric client. It can use either configuration file(s), a Properties object, a typesafe config etc.
monitor {
collectFrequency : 60000ms
tags {
env : prod
app_name: mywebapp
region: us-east-1
cluster: edge1
useHostname: true
}
publishers: [
{
name: Graphite Primary
type: graphite
host: graphiterelay.com
port: 2003
batch_size: 500
sendOnlyRecentlyUpdatedMetrics: true
resetCounters: true
filter.allowList : [
// Only these metrics will be sent through this client.
"*"
]
},
{
name: Prometheus
type: prometheus
sendOnlyRecentlyUpdatedMetrics: false
resetCounters: false
filter.allowList : [
// Only these metrics will be exposed through this client.
"*"
]
relabel: {
// Relabel example. Useful for 3rd party metrics such as those coming from Codahale.
// myapp.db.table1.requests -> myapp_db_requests{db_table="table1"}
"db\\.([^.]+).*": [
{
"db_table": "$1"
}
]
}
}
]
}
// Create metric client.
MetricClient metricClient = new MetricClientFactory()
.create(config.get("monitor.publishers"));
Send metrics
long timeNow = System.currentTimeMillis() / 1000;
metricClient.sendAndReset(metric, timeNow);
metricClient.flush();
Using a metric agent to monitor your metrics
The metric agent monitors a list of metrics. Every 60 seconds (or other configurable frequency), it sends the current metric values to the specified metric client and resets them to zero.
List<Metric> metrics = new ArrayList<>();
// Add metrics to the list
...
MetricAgentConfig config = ImmutableMetricAgentConfig.builder()
.collectFrequency(config.getDuration("monitor.collectFrequency"))
.addMetrics(metrics)
.tags(MetricAgentConfig.tagsFromConfig(config.getConfig("monitor.tags")))
.build();
MetricAgent metricAgent = new MetricAgent(metricClient, config);
metricAgent.startAsync();
...
metricAgent.stopAsync();
Codahale integration
This metrics library is integrated with codahale. You can add one or more codahale MetricRegistry to the metric agent and they will be reported to the backend. Advantages for doing this:
- sends the metrics in batch to the backend
- fail safe (retry + circuit breaker)
- resets the metrics once sent to the backend (useful for Codahale Counters due to https://github.com/dropwizard/metrics/issues/143)
- send to multiple backends in parallel
Build
To build this project:
$ git clone [email protected]:adobe/metrics-client-for-java.git
$ cd metrics-client-for-java/
$ ./gradlew build
Bugs and Feedback
For bugs, questions and discussions please use the GitHub Issues.
LICENSE
Copyright 2018 Adobe Systems Incorporated
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
