log4j-aws-appenders
Appenders for Log4J 1.x, Log4J 2.x and Logback that write to various AWS destinations:
- CloudWatch Logs: AWS-native centralized log management, providing keyword and time range search.
- Kinesis Streams: the first step in a logging pipeline that feeds Elasticsearch and other analytics destinations.
- SNS: useful for real-time error notifications.
In addition to basic log output, this library also provides:
- JsonLayout, which lets you send data to an Elasticsearch/Kibana cluster without the need for parsing.
- JMX integration, which allows the appenders to report operational data.
- Substitions, which allow you to configure the appenders with information from the runtime environment, such as EC2 instance ID.
Usage
To use these appenders, you must add the following libraries to your build:
-
The AWS SDK libraries for whatever destinations and supporting code you need:
- CloudWatch Logs
- Kinesis
- SNS
- IAM (in order to use assumed roles)
- STS (in order to use assumed roles or retrieve current account information)
- EC2 (in order to retrieve instance tags)
- Systems Manager (in order to retrieve values from Parameter Store)
-
The facade library for whatever version of the AWs SDK you're using.
-
The appenders library for your logging framework
Then grant your program the IAM permissions required by your chosen destination(s) and features.
Lastly, configure your logging framework using its configuration mechanism. See the documentation for each appender for the configuration parameters that it uses.
There are example projects that provide typical configurations and Maven POMs.
Versions
I follow the standard MAJOR.MINOR.PATCH
versioning scheme:
MAJOR
is currently 3, representing support for multiple AWS SDKs.MINOR
is incremented for each change that adds signficant functionality or changes the behavior of existing functionality in non-backwards-compatible ways. The API does not break backwards compatibility for minor releases, so your configurations can remain the same.PATCH
is incremented for bugfixes or minor additions to existing features.
Dependencies
The minimum supported dependency versions are:
-
JDK: 1.8
-
Log4J 1.x: 1.2.16
This is the first version that implements
LoggingEvent.getTimeStamp()
, which is needed to order messages when sending to AWS. It's been around since 2010, so if you haven't upgraded already you should. -
Log4J 2.x: 2.10.0
This is the first version that supports custom key/value pairs for
JsonLayout
. If that's not important to you, the library will work with version 2.8 (which introduced a breaking change in backwards compatibility). -
Logback: 1.2.0
This version is required to support
JsonAccessLayout
. If you don't use that, version 1.0.0 will work. -
AWS v1 SDK: 1.11.233
This is the minimum version needed to support all features. As-of this writing it's more than three years old, so if you haven't updated yet you should.
-
AWS v2 SDK: 2.10.0
This is the version that I have used to build and run the integration tests. Earlier versions may work, but I make no guarantees.
To avoid dependency hell, this library does not specify any transitive dependencies. You must explicitly add all required dependencies into your build. I have made an intentional effort to limit dependencies to the bare minimum.
Contributions
At this time I am not accepting contributions. If you find a bug in the code, please submit an issue that explains the problem and provides steps to replicate. Or better, the file and line number where the error exists.
Please note: not being able to specify AWS credentials in your configuration file is not a bug, nor is it an enhancement that I am willing to consider.
For more information
Design and Implementation docs