JarCastingDropwizard Metrics v3 InfluxDb Integration
Support for InfluxDb v0.13 for Dropwizard apps and Metrics v3.1.
metrics-influxdb was copied and backported from Dropwizard metrics which in turn may have started in this project. Some extra features have been added like field filtering and renaming metrics.
dropwizard-metrics-influxdb is a DropWizard metrics reporter factory for building a scheduled reporter for metrics-influxdb.
Usage
Add the dependency to your project:
<dependency>
<groupId>com.izettle</groupId>
<artifactId>dropwizard-metrics-influxdb</artifactId>
<version>${dropwizard-metrics-influxdb.version}</version>
</dependency>
If you have a local influxdb running the bare minimum configuration to put in your dropwizard apps yaml is:
# Metrics reporting
metrics:
reporters:
- type: influxdb
database: mynewdb
After starting your app with this configuration included you'll see some measurements in influxdb:
$ influx
Connected to http://localhost:8086 version 0.9.4.2
InfluxDB shell 0.9.4.2
> use mynewdb
Using database mynewdb
> show measurements
name: measurements
------------------
auth
client_connections
clients
connections
dao
datasources
health
http_server
jvm
jvm_buffers
jvm_classloader
jvm_gc
jvm_memory
jvm_threads
logs
raw_sql
resources
thread_pools
You'll likely want to report to a remote influxdb server though so a typical configuration looks more like this:
# Metrics reporting
metrics:
reporters:
- type: influxdb
protocol: https
host: myinfluxdbhost.com
port: 8086
database: mydb
auth: myuser:mypassword
tags:
host: thehostname
environment: staging
Default configuration
The Dropwizard InfluxDb reporter factory comes with some sensible defaults. Some of the defaults are for reducing storage requirements in the database (1m precision reporting every minute and some default exclusions for example). Others are to get a better fit for the influx model (gauge grouping and measurement mapping).
Measurement/Metric Mappings
Default metric naming that the Metrics library uses does not map particularly well to the InfluxDB tag-based model. This is why we have added some sensible default measurement mappings. For example, the defaults will map all @Timed annotated resource methods whose fully qualified class name includes resource to an influxdb measurement called resources. It will also tag the measurement point with a tag metricName that contains the full metric name. The defaults are for out-of-the-box Dropwizard metrics as well as the recommended Dropwizard project layout.
Tags
Tags for a metric are created by a class implementing the Transform interface configured by tagsTransformer. By default the ClassBasedTransformer is used and it creates tha following tags: metricName, package, className, and method.
Gauge Grouping
Gauge grouping is enabled by default. This will turn a set of metrics into a measurement with separate fields like so:
"org.eclipse.jetty.util.thread.QueuedThreadPool.dw.jobs" : {
"value" : 0
},
"org.eclipse.jetty.util.thread.QueuedThreadPool.dw.size" : {
"value" : 8
},
"org.eclipse.jetty.util.thread.QueuedThreadPool.dw.utilization" : {
"value" : 0.375
},
"org.eclipse.jetty.util.thread.QueuedThreadPool.dw.utilization-max" : {
"value" : 0.0029296875
}
will be grouped to one measurement org.eclipse.jetty.util.thread.QueuedThreadPool.dw:
name: org.eclipse.jetty.util.thread.QueuedThreadPool.dw
-------------------------------------------------------
time jobs size utilization utilization-max
2015-10-22T11:29:00Z 0 13 0.9230769230769231 0.01171875
Since by default we're also mapping the metric to a measurement it will actually be called thread_pools and have a metricName tag with value org.eclipse.jetty.util.thread.QueuedThreadPool.dw:
name: thread_pools
------------------
time jobs size utilization utilization-max metricName
2015-10-26T07:36:14Z 0 13 0.9230769230769231 0.01171875 org.eclipse.jetty.util.thread.QueuedThreadPool.dw
Fields
We default to report the median (p50), the 75th, 99th and 99.9th percentiles and the 1m rate for timers, and just the 1m rate for meters. Since we report every minute the 5 and 15 minute rates can be calculated from the 1 minute rate.
Sender Types
This library can send metrics to InfluxDB directly with http (default), tcp, or udp. In addition to these metrics can also be sent the apps logging facility using logger or to a Kafka topic, see below.
Kafka
Metrics can be passed via Kafka by using the kafka sender type. Example config:
senderType: kafka
database: topic@broker1:9092,broker2:9092
All Defaults
senderType: http
protocol: http
host: localhost
port: 8086
tags: {} # global tags, e.g. environment or host
# push median (p50), some percentiles and the 1m rate
fields:
timers: [p50, p75, p95, p99, p999, m1_rate]
meters: [m1_rate]
groupGauges: yes
# exclude some pre-calculated metrics
excludes:
- ch.qos.logback.core.Appender.debug
- ch.qos.logback.core.Appender.trace
- io.dropwizard.jetty.MutableServletContextHandler.percent-4xx-15m
- io.dropwizard.jetty.MutableServletContextHandler.percent-4xx-1m
- io.dropwizard.jetty.MutableServletContextHandler.percent-4xx-5m
- io.dropwizard.jetty.MutableServletContextHandler.percent-5xx-15m
- io.dropwizard.jetty.MutableServletContextHandler.percent-5xx-1m
- io.dropwizard.jetty.MutableServletContextHandler.percent-5xx-5m
- jvm.attribute.name
- jvm.attribute.vendor
- jvm.memory.heap.usage
- jvm.memory.non-heap.usage
- jvm.memory.pools.Code-Cache.usage
- jvm.memory.pools.Compressed-Class-Space.usage
- jvm.memory.pools.Metaspace.usage
- jvm.memory.pools.PS-Eden-Space.usage
- jvm.memory.pools.PS-Old-Gen.usage
- jvm.memory.pools.PS-Survivor-Space.usage
precision: 1m # only store time precision to the minute
prefix: ""
database: ""
auth: ""
measurementMappings: {}
defaultMeasurementMappings:
health: .*\.health.*
auth: .*\.auth.*
dao: .*\.(jdbi|dao).*
resources: .*\.resources?.*
event_handlers: .*Handler.*
datasources: io\.dropwizard\.db\.ManagedPooledDataSource.*
jobs: .*\..*Job$
clients: org\.apache\.http\.client\.HttpClient.*
client_connections: org\.apache\.http\.conn\.HttpClientConnectionManager.*
connections: org\.eclipse\.jetty\.server\.HttpConnectionFactory.*
thread_pools: org\.eclipse\.jetty\.util\.thread\.QueuedThreadPool.*
logs: ch\.qos\.logback\.core\.Appender.*
http_server: io\.dropwizard\.jetty\.MutableServletContextHandler.*
raw_sql: org\.skife\.jdbi\.v2\.DBI\.raw-sql
jvm: ^jvm$
jvm_attribute: jvm\.attribute.*?
jvm_buffers: jvm\.buffers\..*
jvm_classloader: jvm\.classloader.*
jvm_gc: jvm\.gc\..*
jvm_memory: jvm\.memory\..*
jvm_threads: jvm\.threads.*
# defaults inherited from BaseReporterFactory
durationUnit: MILLISECONDS
rateUnit: SECONDS
# default inherited from MetricsFactory
frequency: 1m
tagsTransformer:
type: ClassBased # default
