YAML config support for Layrry

A Launcher and API for Modularized Java Applications

License

License

Categories

Categories

config Application Layer Libs Configuration
GroupId

GroupId

org.moditect.layrry
ArtifactId

ArtifactId

layrry-config-yaml
Last Version

Last Version

1.0.0.Alpha1
Release Date

Release Date

Type

Type

jar
Description

Description

YAML config support for Layrry
A Launcher and API for Modularized Java Applications

Download layrry-config-yaml

How to add to project

<!-- https://jarcasting.com/artifacts/org.moditect.layrry/layrry-config-yaml/ -->
<dependency>
    <groupId>org.moditect.layrry</groupId>
    <artifactId>layrry-config-yaml</artifactId>
    <version>1.0.0.Alpha1</version>
</dependency>
// https://jarcasting.com/artifacts/org.moditect.layrry/layrry-config-yaml/
implementation 'org.moditect.layrry:layrry-config-yaml:1.0.0.Alpha1'
// https://jarcasting.com/artifacts/org.moditect.layrry/layrry-config-yaml/
implementation ("org.moditect.layrry:layrry-config-yaml:1.0.0.Alpha1")
'org.moditect.layrry:layrry-config-yaml:jar:1.0.0.Alpha1'
<dependency org="org.moditect.layrry" name="layrry-config-yaml" rev="1.0.0.Alpha1">
  <artifact name="layrry-config-yaml" type="jar" />
</dependency>
@Grapes(
@Grab(group='org.moditect.layrry', module='layrry-config-yaml', version='1.0.0.Alpha1')
)
libraryDependencies += "org.moditect.layrry" % "layrry-config-yaml" % "1.0.0.Alpha1"
[org.moditect.layrry/layrry-config-yaml "1.0.0.Alpha1"]

Dependencies

compile (2)

Group / Artifact Type Version
org.moditect.layrry : layrry-config jar 1.0.0.Alpha1
org.yaml : snakeyaml jar 1.27

Project Modules

There are no modules declared in this project.

Layrry - A Launcher and API for Modularized Java Applications

Build Status Download

Latest version: 1.0.0.Alpha1 (announcement)

Layrry is a launcher and Java API for executing modularized Java applications.

It allows to assemble modularized applications based on Maven artifact coordinates of the (modular) JARs to include. Layrry utilizes the Java Module System’s notion of module layers, allowing multiple versions of one module to be used within an application at the same time, as well as dynamically adding and removing modules at application runtime.

The module graph is built either declaratively (using YAML or TOML descriptors) or programmatically (using a fluent API).

Learn more about Layrry in these blog posts:

Layrry also was presented at different (online) conferences and meet-ups, for example at vJUG (slides, recording).

Why Layrry?

The Java Module System doesn’t define any means of mapping between modules (e.g. com.acme.crm) and JARs providing such module (e.g. acme-crm-1.0.0.Final.jar) or retrieving modules from remote repositories using unique identifiers (e.g. com.acme:acme-crm:1.0.0.Final). Instead, it’s the responsibility of the user to obtain all required JARs of a modularized application and provide them via --module-path.

Furthermore, the module system doesn’t define any means of module versioning; i.e. it’s the responsibility of the user to obtain all modules in the right version. Using the --module-path option, it’s not possible, though, to assemble an application that uses multiple versions of one and the same module. This may be desirable for transitive dependencies of an application, which might be required in different versions by two separate direct dependencies.

This is where Layrry comes in: utilizing the notion of module layers, it provides a declarative approach as well as an API for assembling modularized applications, organized in module layers. The JARs to be included are described using Maven GAV (group id, artifact id, version) coordinates, solving the issue of retrieving all required JARs in the right version.

Module layers allow to use different versions of one and the same module in different layers of an application (as long as they are not exposed in a conflicting way on module API boundaries).

Module layers and thus Layrry also allow application extensions to be added and removed dynamically at runtime. Here’s an example for using the Layrry plug-in API to dynamically modify a JavaFX application:

JavaFX app based on Layrry

Using the Layrry Launcher

The Layrry Launcher is a CLI tool which takes a configuration of a layered application and executes it. It’s used like so:

layrry-launcher-1.0.0.Alpha1-all.jar --layers-config  [program arguments]

E.g. like so:

layrry-launcher-1.0.0.Alpha1-all.jar --layers-config hello-world.yml Alice Bob

The application layers configuration file is a YAML file which the following structure:

layers:
  <name 1>:
    modules:
      - "G:A:V"
      - "G:A:V"
      - ...
  <name 2>:
    parents:
      - "<name 1>"
    modules:
      - ...
      - ...
  <name 3>:
    parents:
      - "<name 2"
    directory: "relative/path/to/directory/of/layer/directories

main:
  module: <main module>
  class: <main class>

Each layer comprises:

  • A unique name

  • The list of parent layers

  • The list of contained modules given via Maven GAV coordinates OR

  • A directory which contains one or more sub-directories, each of which represent one layer made up of the modular JARs within that sub-directory; the directory path is resolved relatively to the location of the layrry.yml file. Alternatively the directory may be an absolute path however be very careful as this may cause a non portable configuration.

As an example, consider the following application whose modules foo and bar depend on two different versions of the greeter module:

Layrry Example

Running this application wouldn’t be possible with the default module path, which only allows for one version of a given module. Here is how the application can be executed via Layrry, organizing all the modules in multiple layers:

layers:
  log:
    modules:
      - "org.apache.logging.log4j:log4j-api:2.14.0"
      - "org.apache.logging.log4j:log4j-core:2.14.0"
      - "com.example:logconfig:1.0.0"
  foo:
    parents:
      - "log"
    modules:
      - "com.example:greeter:1.0.0"
      - "com.example:foo:1.0.0"
  bar:
    parents:
      - "log"
    modules:
      - "com.example:greeter:2.0.0"
      - "com.example:bar:1.0.0"
  app:
    parents:
      - "foo"
      - "bar"
    modules:
      - "com.example:app:1.0.0"
main:
  module: com.example.app
  class: com.example.app.App

Alternatively you may use TOML instead of YAML

[layers.log]
  modules = [
    "org.apache.logging.log4j:log4j-api:2.14.0",
    "org.apache.logging.log4j:log4j-core:2.14.0",
    "com.example.it:it-logconfig:1.0.0"]
[layers.foo]
  parents = ["log"]
  modules = [
    "com.example.it:it-greeter:1.0.0",
    "com.example.it:it-foo:1.0.0"]
[layers.bar]
  parents = ["log"]
  modules = [
    "com.example.it:it-greeter:2.0.0",
    "com.example.it:it-bar:1.0.0"]
[layers.app]
  parents = ["foo", "bar"]
  modules = ["com.example.it:it-app:1.0.0"]
[main]
  module = "com.example.app"
  class = "com.example.app.App"

Be sure to use .toml as file extension to let Layrry know which format should be parsed.

You can find the complete example in the tests of the Layrry project.

The Layrry Launcher accepts the following arguments:

  • --basedir: The base directory from which plugin directories will be resolved. Layrry will use the parent directory of the layers config file if this value is not set.

  • --layers-config: Path to the layers config file. The file must use any of the supported config formats. REQUIRED.

  • --properties: Path to additional properties in Java .properties format. These properties will be used to replace value placeholders found in the layers config file. OPTIONAL.

Using JBang

JBang can launch self contained Java sources, JShell scripts, JARs. jbang has a feature that allows you to try out Layrry without having to install or build Layrry yourself. You only need a JDK (11+ is preferred) and jbang installed. Once you do, you may invoke the previous example with

jbang layrry@moditect --layers-config layers.yml

JBang will resolve and download the appropriate Layrry bootstrap binary, then Layrry resolves the modules described in the input configuration file, finally the application is launched.

Dynamic Plug-Ins

Layrry also supports the dynamic addition and removal of plug-ins at runtime. For that, simply add or remove plug-in sub-directories to the directory of a layer configuration. Layrry watches the given plug-ins directory and will add or remove the corresponding module layer to/from the application in case a new plug-in is added or removed. The core of an application can react to added or removed module layers. In order to do so, the module org.moditect.layrry:layrry-platform must be added to the application core layer and an implementation of the PluginLifecycleListener interface must be created and registered as service:

public interface PluginLifecycleListener {
    void pluginAdded(PluginDescriptor plugin);

    void pluginRemoved(PluginDescriptor plugin);
}

Typically, an application will retrieve application-specific services from newly added module layers:

@Override
public void pluginAdded(PluginDescriptor plugin) {
  ServiceLoader<MyService> services = ServiceLoader.load(
      plugin.getModuleLayer(), MyService.class);

    services.forEach(service -> {
      // only process services declared by the added layer itself, but not
      // from ancestor layers
      if (service.getClass().getModule().getLayer() == layer) {
        // process service ...
      }
    });
}

To avoid class-loader leaks, it’s vital that all references to plug-in contributed classes are released upon pluginRemoved(). Note that classes typically will not instantly be unloaded, but only upon the next full GC (when using G1).

You can find a complete example for the usage of dynamic plug-ins in the vertx-example directory: "Layrry Links" is an example application for managing golf courses, centered around a web application core built using Vert.x. Routes of the web application (/members, /tournaments) are contributed by plug-ins which can be added to or removed from the application at runtime. The routes path shows all routes available at a given time.

Plugins may be packaged in 3 ways:

  1. As a single JAR file. No nested JARs are allowed.

  2. As a Zip file. Multiple JARs may be packaged.

  3. As a Tar(.gz) file. Multiple JARs may be packaged.

For Zip and Tar packages, the use of a root entry matching the name of containing file is permited, however it’s preferred if said root entry were omitted. Some examples:

Single JAR
plugin-1.0.jar
 |- com
 |- com/acme
 |- com/acme/Plugin.class
 \- module-info.class
Plain Zip (or Tar)
plugin-1.0.zip
 |- plugin-1.0.jar
 |- dependency-foo-1.0.0.jar
 \- dependency-bar-1.0.0.jar
Root Zip (or Tar)
plugin-1.0.zip
 |- plugin-1.0
 |- plugin-1.0/plugin-1.0.jar
 |- plugin-1.0/dependency-foo-1.0.0.jar
 \- plugin-1.0/dependency-bar-1.0.0.jar

Parameterized Layer Configuration

Layrry supports the Mustache template syntax, enabling parameterization of the content found in configuration files, regardless of their target format (YAML, TOML, etc). To use this feature you must use a {{property}} expression to refer to value placeholders. Layrry makes all System properties available for value replacement, as well as an extra set of properties that are related to OS values; these include all properties exposed by the os-maven-plugin. If the --properties command flag is passed to the Layrry Launcher then all properties found in the given properties file will also become available.

Additionally, Layrry resolves the following properties

  • os.detected.jfxname: specific to JavaFX. Values may be one of linux, win, mac.

  • os.detected.lwjglname: specific to LWJGL. Values may be one of linux, linux-arm32, windows, windows-x86, macosx.

The following example shows a parameterized TOML config file for a JavaFX application that can be run on any of the 3 platforms supported by JavaFX

.layers.toml
[layers.javafx]
    modules = [
        "org.openjfx:javafx-base:jar:{{os.detected.jfxname}}:{{javafx_version}}",
        "org.openjfx:javafx-controls:jar:{{os.detected.jfxname}}:{{javafx_version}}",
        "org.openjfx:javafx-graphics:jar:{{os.detected.jfxname}}:{{javafx_version}}",
        "org.openjfx:javafx-web:jar:{{os.detected.jfxname}}:{{javafx_version}}",
        "org.openjfx:javafx-media:jar:{{os.detected.jfxname}}:{{javafx_version}}"]
[layers.core]
    modules = [
        "org.kordamp.tiles:modular-tiles-model:{{project_version}}",
        "org.kordamp.tiles:modular-tiles-core:{{project_version}}",
        "org.kordamp.tiles:modular-tiles-app:{{project_version}}",
        "org.moditect.layrry:layrry-platform:{{layrry_version}}",
        "eu.hansolo:tilesfx:{{tilesfx_version}}"]
    parents = ["javafx"]
[layers.plugins]
    parents = ["core"]
    directory = "plugins"
[main]
  module = "org.kordamp.tiles.app"
  class = "org.kordamp.tiles.app.Main"
versions.properties
project_version = 1.0.0
javafx_version = 11.0.2
tilesfx_version = 11.44
layrry_version = 1.0.0.Alpha1

This application can be launched as

layrry-launcher-1.0.0.Alpha1-all.jar --layers-config layers.toml --properties versions.properties

Remote Configuration

Layrry supports loading external configuration files (inputs to --layers-config and --properties) both from local and remote sources. For example, the previous layers.toml and versions.properties files could be accessed from a remote server that exposes those resources via HTTPS, such as

layrry-launcher-1.0.0.Alpha1-all.jar \
  --basedir /home/user/joe \
  --layers-config https://server:port/path/to/layers.toml \
  --properties https://server:port/path/to/versions.properties

It’s important to note that setting the --basedir config flag is more important when remote layer configuration is in use, as that ensures plugin directories will be resolved from the same location, otherwise the basedir location will be inferred as System.getProperty("user.dir") which may produce unexpected results when invoked from different locations.

Plugin directories are always local, even if defined in remote layer configuration files. You may mix remote and local resources as you deem necessary, that is, the following combinations are valid:

All remote
layrry-launcher-1.0.0.Alpha1-all.jar \
  --basedir /home/user/joe \
  --layers-config https://server:port/path/to/layers.toml \
  --properties https://server:port/path/to/versions.properties
All local
layrry-launcher-1.0.0.Alpha1-all.jar \
  --basedir /home/user/joe \
  --layers-config layers.toml \
  --properties versions.properties
Mixed
layrry-launcher-1.0.0.Alpha1-all.jar \
  --basedir /home/user/joe \
  --layers-config https://server:port/path/to/layers.toml \
  --properties versions.properties
layrry-launcher-1.0.0.Alpha1-all.jar \
  --basedir /home/user/joe \
  --layers-config layers.toml \
  --properties https://server:port/path/to/versions.properties

Proxy Configuration

You may need to configure a proxy when using the remote configuration feature. The following properties may be used to configure a proxy:

Key Description

use.proxy

Whether to use any proxy or not. Defaults to false.

http.proxy

Whether to use HTTP proxy or not. Defaults to false.

http.proxyHost

Defaults to empty String.

http.proxyport

Defaults to 80.

http.proxyUser

Defaults to empty String.

http.proxyPassword

Defaults to empty String.

http.nonProxyHosts

Defaults to localhost|127.*|[::1].

https.proxy

Whether to use HTTPS proxy or not. Defaults to false.

https.proxyHost

Defaults to empty String.

https.proxyport

Defaults to 443.

https.proxyUser

Defaults to empty String.

https.proxyPassword

Defaults to empty String.

socks.proxy

Whether to use SOCKS proxy or not. Defaults to false.

socksProxyHost

Defaults to empty String.

socksProxyPort

Defaults to 1080.

socks.proxyUser

Defaults to empty String.

socks.proxyPassword

Defaults to empty String.

These properties may be set as System properties by passing -Dkey=value as part of command line arguments when using the launcher, or as part of the additional properties file if --properties is given as an argument to the launcher.

Using the Layrry API

In addition to the YAML-based/TOML-based launcher, Layrry provides also a Java API for assembling and running layered applications. This can be used in cases where the structure of layers is only known at runtime, or for implementing plug-in architectures.

In order to use Layrry programmatically, add the following dependency to your pom.xml:

<dependency>
    <groupId>org.moditect.layrry</groupId>
    <artifactId>layrry</artifactId>
    <version>1.0.0.Alpha1</version>
</dependency>

Then, the Layrry Java API can be used like this (showing the same example as above):

Layers layers = Layers.builder()
    .layer("log")
        .withModule("org.apache.logging.log4j:log4j-api:2.14.0")
        .withModule("org.apache.logging.log4j:log4j-core:2.14.0")
        .withModule("com.example:logconfig:1.0.0")
    .layer("foo")
        .withParent("log")
        .withModule("com.example:greeter:1.0.0")
        .withModule("com.example:foo:1.0.0")
    .layer("bar")
        .withParent("log")
        .withModule("com.example:greeter:2.0.0")
        .withModule("com.example:bar:1.0.0")
    .layer("app")
        .withParent("foo")
        .withParent("bar")
        .withModule("com.example:app:1.0.0")
    .build();

layers.run("com.example.app/com.example.app.App", "Alice");

Configuring Artifact Resolution

Layrry relies on Maven’s API to resolve artifacts. By default, Maven Local, Maven Central and every other setting configured at ~/.m2/settings.xml are available to Layrry. You can tweak and configure those settings by editing the ~/.m2/settings.xml file. Alternatively you may instruct Layrry to use a different configuration file, skip querying Maven Central, or stop all resolutions via remote repositories.

Disable All Remote Maven Repositories

Java
Layers layers = Layers.builder()
    .resolve(Resolvers.remote().workOffline(true))
    .layer(...)
Yaml
resolve:
  workOffline: true
  ...
Toml
[resolve]
  workOffline = true
  ...

Disable All Remote and Local Maven Repositories

Java
Layers layers = Layers.builder()
    .resolve(Resolvers.remote().enabled(false))
    .layer(...)
Yaml
resolve:
  remote: false
  ...
Toml
[resolve]
  remote = false
  ...

Use Alternate Maven Settings File

Java
Layers layers = Layers.builder()
    .resolve(Resolvers.remote()
         .fromFile(Paths.get("/path/to/settings.xml")))
    .layer(...)
Yaml
resolve:
  fromFile: "/path/to/settings.xml"
  ...
Toml
[resolve]
  fromFile = "/path/to/settings.xml"
  ...

Local Artifact Resolution

Layrry can resolve artifacts from additional local sources. These sources must follow specific layouts for organizing artifacts. Currently flat and default layouts are supported, which are provided by Maven and Gradle plugins. Local repositories will always be queried first, then any remote repositories if available.

Flat Layout

This layout organizes all artifacts in a single directory, for example

repodir
 |-- foo-1.0.0.jar
 \-- bar-2.0.0.jar

Default Layout

This layout organizes all artifacts following the Maven coordinates conventions, for example

repodir
  |-- com
  |    \-- acme
  |        \-- foo
  |            \-- 1.0.0
  |                \-- foo-1.0.0.jar
  \-- org
       \-- random
           \-- bar
               \-- 2.0.0
                   \-- bar-2.0.0.jar

Use Local Repositories

Java
Layers layers = Layers.builder()
    .resolve(Resolvers.local()
        .withLocalRepo("repoName", Paths.get("/path/to/repository/directory").toAbsolutePath(), "flat"))
    ...
Yaml
resolve:
  localRepositories:
    repoName:
      layout: "flat"
      path: "/path/to/repository/directory"
Toml
[resolve.localRepositories.repoName]
  layout = "flat"
  path   = "/path/to/repository/directory"

The path may be absolute as shown in the examples or relative, in which it will be resolved relative to the config file path.

Building Layrry

Layrry can be built from source by running the following command

$ mvn install

Java 11 or later is needed in order to do so.

Contributing

Your contributions to Layrry are very welcomed. Please open issues with your feature suggestions as well as pull requests. Before working on larger pull requests, it’s suggested to reach out to @gunnarmorling.

License

Layrry is licensed under the Apache License version 2.0.

org.moditect.layrry

ModiTect

A family of open-source projects related to Java Modules, Flight Recorder, and more

Versions

Version
1.0.0.Alpha1