JarCastingThis project has moved to the Spring Cloud Incubator.
Spring Cloud Kubernetes
Spring Cloud integration with Kubernetes
Features
- DiscoveryClient for Kubernetes
- KubernetesClient autoconfiguration
- PropertySource
- ConfigMap PropertySource
- Secrets PropertySource
- PropertySource Reload
- Pod Health Indicator
- Transparency (its transparent wether the code runs in or outside of Kubernetes)
- Kubernetes Profile Autoconfiguration
- Ribbon discovery in Kubernetes
- Zipkin discovery in Kubernetes
- ConfigMap Archaius Bridge
DiscoveryClient for Kubernetes
This project provides an implementation of Discovery Client for Kubernetes. This allows you to query Kubernetes endpoints (see services) by name. This is something that you get for free just by adding the following dependency inside your project:
<dependency>
<groupId>io.fabric8</groupId>
<artifactId>spring-cloud-starter-kubernetes</artifactId>
<version>${latest.version}</version>
</dependency>
Then you can inject the client in your cloud simply by:
@Autowire
private DiscoveryClient discoveryClient;
If for any reason you need to disable the DiscoveryClient you can simply set the following property:
spring.cloud.kubernetes.discovery.enabled=false
Some spring cloud components use the DiscoveryClient in order obtain info about the local service instance. For this to work you need to align the service name with spring.application.name.
Kubernetes PropertySource
The most common approach to configure your spring boot application is to edit the application.yaml file. Often the user may override properties by specifying system properties or env variables.
ConfigMap PropertySource
Kubernetes has the notion of ConfigMap for passing configuration to the application. This project provides integration with ConfigMap to make config maps accessible by spring boot.
The ConfigMap PropertySource when enabled will lookup Kubernetes for a ConfigMap named after the application (see spring.application.name). If the map is found it will read its data and do the following:
- apply individual configuration properties.
- apply as yaml the content of any property named
application.yaml - apply as properties file the content of any property named
application.properties
Example:
Let's assume that we have a spring boot application named demo that uses properties to read its thread pool configuration.
pool.size.corepool.size.maximum
This can be externalized to config map in yaml format:
kind: ConfigMap
apiVersion: v1
metadata:
name: demo
data:
pool.size.core: 1
pool.size.max: 16
Individual properties work fine for most cases but sometimes we yaml is more convinient. In this case we will use a single property named application.yaml and embed our yaml inside it:
kind: ConfigMap
apiVersion: v1
metadata:
name: demo
data:
application.yaml: |-
pool:
size:
core: 1
max:16
Notes:
-
To access ConfigMaps on OpenShift the service account needs at least view permissions i.e.:
oc policy add-role-to-user view system:serviceaccount:$(oc project -q):default -n $(oc project -q)
Secrets PropertySource
Kubernetes has the notion of Secrets for storing sensitive data such as password, OAuth tokens, etc. This project provides integration with Secrets to make secrets accessible by spring boot.
The Secrets PropertySource when enabled will lookup Kubernetes for Secrets from the following sources:
- reading recursively from secrets mounts
- named after the application (see
spring.application.name) - matching some labels
Please note that by default, consuming Secrets via API (points 2 and 3 above) is not enabled.
If the secrets are found theirs data is made available to the application.
Example:
Let's assume that we have a spring boot application named demo that uses properties to read its ActiveMQ and PostreSQL configuration.
amq.usernameamq.passwordpg.usernamepg.password
This can be externalized to Secrets in yaml format:
-
ActiveMQ
apiVersion: v1 kind: Secret metadata: name: activemq-secrets labels: broker: activemq type: Opaque data: amq.username: bXl1c2VyCg== amq.password: MWYyZDFlMmU2N2Rm
-
PostreSQL
apiVersion: v1 kind: Secret metadata: name: postgres-secrets labels: db: postgres type: Opaque data: amq.username: dXNlcgo= amq.password: cGdhZG1pbgo=
You can select the Secrets to consume in a number of ways:
-
By listing the directories were secrets are mapped:
-Dspring.cloud.kubernetes.secrets.paths=/etc/secrets/activemq,etc/secrets/postgresIf you have all the secrets mapped to a common root, you can set them like:
-Dspring.cloud.kubernetes.secrets.paths=/etc/secrets -
By setting a named secret:
-Dspring.cloud.kubernetes.secrets.name=postgres-secrets -
By defining a list of labels:
-Dspring.cloud.kubernetes.secrets.labels.broker=activemq -Dspring.cloud.kubernetes.secrets.labels.db=postgres
Properties:
| Name | Type | Default | Description |
|---|---|---|---|
| spring.cloud.kubernetes.secrets.enabled | Boolean | true | Enable Secrets PropertySource |
| spring.cloud.kubernetes.secrets.name | String | ${spring.application.name} | Sets the name of the secret to lookup |
| spring.cloud.kubernetes.secrets.labels | Map | null | Sets the labels used to lookup secrets |
| spring.cloud.kubernetes.secrets.paths | List | null | Sets the paths were secrets are mounted /example 1) |
| spring.cloud.kubernetes.secrets.enableApi | Boolean | false | Enable/Disable consuming secrets via APIs (examples 2 and 3) |
Notes:
- The property spring.cloud.kubernetes.secrets.labels behave as defined by Map-based binding.
- The property spring.cloud.kubernetes.secrets.paths behave as defined by Collection-based binding.
- Access to secrets via API may be restricted for security reasons, the preferred way is to mount secret to the POD.
PropertySource Reload
Some applications may need to detect changes on external property sources and update their internal status to reflect the new configuration. The reload feature of Spring Cloud Kubernetes is able to trigger an application reload when a related ConfigMap or Secret change.
This feature is disabled by default and can be enabled using the configuration property spring.cloud.kubernetes.reload.enabled=true (eg. in the application.properties file).
The following levels of reload are supported (property spring.cloud.kubernetes.reload.strategy):
- refresh (default): only configuration beans annotated with
@ConfigurationPropertiesor@RefreshScopeare reloaded. This reload level leverages the refresh feature of Spring Cloud Context. - restart_context: the whole Spring ApplicationContext is gracefully restarted. Beans are recreated with the new configuration.
- shutdown: the Spring ApplicationContext is shut down to activate a restart of the container. When using this level, make sure that the lifecycle of all non-daemon threads is bound to the ApplicationContext and that a replication controller or replica set is configured to restart the pod.
Example:
Assuming that the reload feature is enabled with default settings (refresh mode), the following bean will be refreshed when the config map changes:
@Configuration
@ConfigurationProperties(prefix = "bean")
public class MyConfig {
private String message = "a message that can be changed live";
// getter and setters
}
A way to see that changes effectively happen is creating another bean that prints the message periodically.
@Component
public class MyBean {
@Autowired
private MyConfig config;
@Scheduled(fixedDelay = 5000)
public void hello() {
System.out.println("The message is: " + config.getMessage());
}
}
The message printed by the application can be changed using a config map like the following one:
apiVersion: v1
kind: ConfigMap
metadata:
name: reload-example
data:
application.properties: |-
bean.message=Hello World!
Any change to the property named bean.message in the Config Map associated to the pod will be reflected in the output of the program (more details here about how to associate a Config Map to a pod).
The full example is available in spring-cloud-kubernetes-reload-example.
The reload feature supports two operating modes:
- event (default): watches for changes in config maps or secrets using the Kubernetes API (web socket). Any event will produce a re-check on the configuration and a reload in case of changes. The
viewrole on the service account is required in order to listen for config map changes. A higher level role (eg.edit) is required for secrets (secrets are not monitored by default). - polling: re-creates the configuration periodically from config maps and secrets to see if it has changed. The polling period can be configured using the property
spring.cloud.kubernetes.reload.periodand defaults to 15 seconds. It requires the same role as the monitored property source. This means, for example, that using polling on file mounted secret sources does not require particular privileges.
Properties:
| Name | Type | Default | Description |
|---|---|---|---|
| spring.cloud.kubernetes.reload.enabled | Boolean | false | Enables monitoring of property sources and configuration reload |
| spring.cloud.kubernetes.reload.monitoring-config-maps | Boolean | true | Allow monitoring changes in config maps |
| spring.cloud.kubernetes.reload.monitoring-secrets | Boolean | false | Allow monitoring changes in secrets |
| spring.cloud.kubernetes.reload.strategy | Enum | refresh | The strategy to use when firing a reload (refresh, restart_context, shutdown) |
| spring.cloud.kubernetes.reload.mode | Enum | event | Specifies how to listen for changes in property sources (event, polling) |
| spring.cloud.kubernetes.reload.period | Long | 15000 | The period in milliseconds for verifying changes when using the polling strategy |
Notes:
- Properties under spring.cloud.kubernetes.reload.* should not be used in config maps or secrets: changing such properties at runtime may lead to unexpected results;
- Deleting a property or the whole config map does not restore the original state of the beans when using the refresh level.
Pod Health Indicator
Spring Boot uses HealthIndicator to expose info about the health of an application. That makes it really useful for exposing health related information to the user and are also a good fit for use as readiness probes.
The Kubernetes health indicator which is part of the core modules exposes the following info:
- pod name
- visible services
- flag that indicates if app is internal or external to Kubernetes
Transparency
All of the features described above will work equally fine regardless of wether our application is inside Kubernetes or not. This is really helpful for development and troubleshooting.
Kubernetes Profile Autoconfiguration
When the application is run inside Kubernetes a profile named kubernetes will automatically get activated. This allows the user to customize the configuration that will be applied in and out of kubernetes (e.g. different dev and prod configuration).
Ribbon discovery in Kubernetes
A Kubernetes based ServerList for Ribbon has been implemented. The implementation is part of the spring-cloud-kubernetes-ribbon module and you can use it by adding:
<dependency>
<groupId>io.fabric8</groupId>
<artifactId>spring-cloud-starter-kubernetes-netflix</artifactId>
<version>${latest.version></version>
</dependency>
The ribbon discovery client can be disabled by setting spring.cloud.kubernetes.ribbon.enabled=false.
By default the client will detect all endpoints with the configured client name that lives in the current namespace. If the endpoint contains multiple ports, the first port will be used. To fine tune the name of the desired port (if the service is a multiport service) or fine tune the namespace you can use one of the following properties.
KubernetesNamespacePortName
Examples that are using this module for ribbon discovery are:
Zipkin discovery in Kubernetes
Zipkin is a distributed tracing system and it is also supported by Sleuth.
Discovery of the services required by Zipkin (e.g. zipkin-query) is provided by spring-cloud-kubernetes-zipkin module and you can use it by adding:
<dependency>
<groupId>io.fabric8</groupId>
<artifactId>spring-cloud-starter-kubernetes-zipkin</artifactId>
<version>${latest.version></version>
</dependency>
This works as an extension of spring-cloud-sleuth-zipkin.
Examples of application that are using Zipkin discovery in Kubernetes:
ConfigMap Archaius Bridge
Section ConfigMap PropertySource provides a brief explanation on how to configure spring boot application via ConfigMap. This approach will aid in creating the configuration properties objects that will be passed in our application. If our application is using Archaius it will indirectly benefit by it. An alternative approach that provides more direct Archaius support without getting in the way of spring configuration properties by using spring-cloud-kubernetes-archaius that is part of the Netflix starter.
This module allows you to annotate your application with the @ArchaiusConfigMapSource and archaius will automatically use the configmap as a watched source (get notification on changes).
Troubleshooting
Namespace
Most of the components provided in this project need to know the namespace. For Kubernetes (1.3+) the namespace is made available to pod as part of the service account secret and automatically detected by the client. For earlier version it needs to be specified as an env var to the pod. A quick way to do this is:
env:
- name: "KUBERNETES_NAMESPACE"
valueFrom:
fieldRef:
fieldPath: "metadata.namespace"
Service Account
For distros of Kubernetes that support more fine-grained role-based access within the cluster, you need to make sure a pod that runs with spring-cloud-kubernetes has access to the Kubernetes API. For example, OpenShift has very comprehensive security measures that are on by default (typically) in a shared cluster. For any service accounts you assign to a deployment/pod, you need to make sure it has the correct roles. For example, you can add cluster-reader permissions to your default service account depending on the project you're in:
oc policy add-role-to-user cluster-reader system:serviceaccount:<project/namespace>:default
Building
You can just use maven to build it from sources:
mvn clean install
Usage
The project provides a "starter" module, so you just need to add the following dependency in your project.
<dependency>
<groupId>io.fabric8</groupId>
<artifactId>spring-cloud-starter-kubernetes</artifactId>
<version>x.y.z</version>
</dependency>
