Spring Cloud Kubernetes ConfigWatcher
New approach to auto reloading.
We all may have come across the situation where we need to manage application specific properties by externalizing the configuration via KV store like Harshi Corp Consul or Spring Cloud Config etc. The expectation was always to manage these properties externally from the application by storing it at some repository and make it available during the container initialization process.
This was all good till the time container is initialized. The challenges come when changes are required at runtime and the changed properties need to he injected to the application. In a microservice architecture not every component behaves the same. Some probably will have a different set of needs than the other. The traditional solutions that we have been using is to restart the containers. Depending upon the tool you use, you probably may or may not have to do this. Restart initiates the init process again and pulls the latest properties from the Config store. What if we don’t have the luxury to restart the containers and what if these configurations need to be pushed to the containers at runtime? Some applications will even have the requirement to detect the changes on external property source and update the internal status to reflect the new changes. Considering this article is specifically dedicated to Spring-Cloud-Kubernetes project, let’s see how it can be done using Kubernetes native resource like ConfigMaps and Secret and eliminate the need to having any external or third party tools for config management.
Spring-Cloud-Kubernetes till 2020.0
Spring-Cloud-Kubernetes till 2020.0 release has been handling this declaratively. With Spring-Cloud-Kubernetes you have the option to use the Kubernetes resources like ConfigMaps and Secrets to manage our configurations. Using ConfigMaps and Secrets you can externalize the parameters to pass to your application in the form of key-value pair or embed the entire application.properties or application.yml file. The biggest advantage of using these native services on Spring-Cloud-Kubernetes was the hot reloading of the changes made to the application containers. The entire setup of utilizing the native resources and reloading was achieved by Injecting the dependencies associated to the feature and letting the component know that config is enabled, at which namespace it is available, and the name of ConfigMaps or Secret.
STEP 1
STEP 2
STEP 3
STEP 4
Back then the only must have requirement to use this native service was to enable it and make sure the name of the ConfigMap matches the name of the application. We also had to specify the configmaps namespace etc. in the application bootstap or properties files. This was my least favorite setting as I never like the idea of embedding my infrastructure specific information inside the application. Definitely the other requirement specific to Role and RoleBinding still remains the same which lets the DiscoveryClient implemented by the application privileges scan resources running on other namespaces. The reload feature was disabled by default. It was enabled by setting its properties to true which basically leverages the refresh feature of Spring Cloud context.
The other two reload option available was restart_context and shutdown.
The reload feature has support for two modes :-
- Event :: Which is the default mode where it Watches for the changes on the Configmap or secrets by using the WebSockets of Kubernetes API. Any Event produces a re-check on the configuration and on detecting the change reload is triggered. This is where the Role setup comes to rescue where a view role set on the service account was required in order to listen for the config map changes.
- Polling :: You have the option to set the polling period that periodically recreates the configuration from config map and secrets to see if it has changed.
Spring-Cloud-Kubernetes Post 2020.0
So this was all well till 2020.0 and then the reload feature was deprecated. Welcome Configuration Watcher. This new feature is the hybrid form of earlier version. This new controller has the capability to send refresh notification to applications in two ways :-
- Using HTTP where application needs to be exposed with /refresh actuator endpoint and should be accessible from within the cluster.
- Using Spring Cloud Bus with the help of message broker running inside the cluster.
Everything from the setup perspective remained the same except that :-
You now don’t have to tell the application about the Configmap’s Metadata
Spring Cloud provides the ability to refresh the application context without restart of the application. This was done by hitting the actuator endpoint /refresh or by publishing an event via RefreshRemoteApplicationEvent using Spring Cloud Bus.
So let’s see how the configuration refresh of Spring Cloud on Kubernetes can be achieved. We need a mechanism between our ConfigMap services that can reload the context of our application once the change is detected at the ConfigMap. We also need to let that mechanism know which ConfigMap events to subscribe to and which services need to be updated.
That mechanism is the Spring Cloud Kubernetes Configuration Watcher Controller
The Controller needs access to ConfigMap, Pods, Services, Secrets, and Endpoints in Kubernetes Cluster to act properly. It will react to the changes in ConfigMap and Secrets if the Configmap or Secrets has been set to label of:-
For secret the properties is:
If the values is set to false or if the labels are missing from ConfigMap or Secrets then the changes will be ignored. If the values are set to true then Configuration Watcher will take the name of the ConfigMap or Secret and will send a notification to the application available with the same name. The naming process of ConfigMap and its associated Service remains the same. They need to be same.
Following is how Configwatchers deployment config looks like. You can have any flavors of its deployment based on your need for example, Config watcher per namespace, per node etc.
Run your application with the config highlighted and deploy Configwatcher as a service and that’s what it takes from the operation perspective. The Demo Service used as an example in this article has a scheduler that runs every 5 seconds and reads the config data. This Config bean is updated whenever It receives an update event back from ConfigWatcher.
Configuration Watcher has really simplified the reload mechanism and has also unbinded Kubernetes resources mapping from within the application. With everything configured and in place you can deploy and edit the properties deployed in Configmap and the Configuration Watcher will take care of reloading them to the application.
A working example can be found at ConfigWatch
