Spring Cloud Kubernetes Persistence
Adding persistence to Spring Cloud Kubernetes
This is the next article in the series and so like before the assumption will be that you are familiar with Spring Boot and Kubernetes. This series of articles is completely dedicated to the new project from Spring.io known as Spring Cloud Kubernetes. In the previous article, I have given the conceptual idea of what the architecture of the application looks like. If you have not gone through that then you can refer to the following link to get an idea on what and why.
https://amit4aws.medium.com/spring-cloud-kubernets-d542fe1b3bd5
Compared to regular Spring boot application deployment architecture we are not doing anything fancy here. The goal is to see how things are different and are done when using Spring Cloud Kubernetes. The example application is built on Spring-Cloud-Kubernetes project. It contains two spring boot business components deployed in two different namespaces. They communicate with each other via Spring-Cloud-Kubernetes DiscoveryClient over Feign. Compare to regular spring boot implementation, we don’t have Eureka for service discovery but we are relying on the Spring Cloud Kubernetes Discovery Client to do the service discovery across different namespace.
Now continuing this article, Application properties holds configurable properties for our application. To manage application specific properties we are externalizing the configuration via KV store like Harshi Corp Consul or Spring Cloud Config etc. so the application specific properties can be remotely managed and easily deployed during runtime. Similar to these options, Kubernetes provides a resource named ConfigMap to externalize the parameters to pass to your application in the form of key-value pairs or via embedded application.properties or application.yaml files. Considering we are discussing about a spring cloud kubernetes project then why not utilize the resource we already have in place than introducing anything new to the architecture.
Our new revised architecture of the application looks like this :-
The Connections
- All pods (business components & DB) run in their own namespace
- They are assigned a serviceaccount
- These service accounts are backed by Cluster Admin Role which helps them query Kube Admin for DB or Component to Component Interaction.
We are introducing a new component to our infrastructure in DB namespace. MySQL db in this example is used as a docker container managing the DB data for the microservices. As you can see we have used Configmap and Secret here as well, but there is nothing fancy as it’s the standard and has a very basic way of DB initialization where properties are managed and provided to the deployment for the application on kubernetes infrastructure.
The two business component also have ConfigMaps and Secrets. As per the architecture, I have kept the ConfigMap and Secret separate for each component in their own namespace. So we have two different ways of utilizing the configuration resources in this example. The standard or the native way which MySQL DB demonstrates and via Spring Cloud Kubernetes way that business components are using.
Let’s see how these three components are interacting with each other and utilizing the resources.
There is no spring cloud feature involved here. Secrets and Configurations associated to Mysql are stored in Kubernetes Secrets and ConfigMap which are pulled as environment variables. A Persistence volume is dedicated for its Storage class as well.
Second Hop:- Business Components
The business Component are utilizing the following process:-
- Pull dependencies associated to spring-cloud-starter-kubernetes-config
- Dependencies associated to MySQL. Since we are utilizing JPA we will be needing Its dependencies along with MySQL connector
- Isolated the configuration of Init and application properties to two different properties. This was required because there are things business component will need during initialization and to execute steps post initialization.
bootstrap.yml { To take care of things during bootstrap}
application.yml { To take care of things during post initialization }
Bootstrap holds the configuration to make the application discoverable across different namespace. This was done by setting the cloud::kubernetes::discovery::all-namespaces to true. Expose your actuator endpoints. Not a good idea for production but you have the option to be selective at which endpoints need to be exposed. This is required for the config-watcher to hot reload your container with latest configuration from configmap. You also need to enable the endpoint restart properties. This is required as well for the context reload on config refresh by configuration watcher.
- The entire application.yml configuration which includes :-
DB initialization parameters like JDBC URL,hostname, passwords etc.
Config Watcher properties
Logging etc
Is embedded inside the configmap.
As we can see our Item Service business component holds minimum configuration in bootstap yml. It is only making the component discoverable across the namespace. The design change that we are introducing here is moving the entire application yaml configuration properties to ConfigMap. The application configuration properties that holds other configuration reside inside the application yaml which is embedded in to configmap. Spring boot is able to load application configuration during Init because of the following dependencies :-
DB host, DB name, DB username and DB password were passed as a env properties reference from Configmap inside kubernetes deployment script. This was required because the application yaml has it embedded. There may be better way of achieving this, but I used this approach for my use case. We also have a Init SQL in our config that will fire the SQL to load the Item data to the Mysql DB at run time.
Third Hop:- Business Components Deployment Script ::
Application configuration has Mysql DB Parameters defined
and I am initializing it via env variables set at the deployment script which eventually is pulling the details from the mapped configmap.
Last Hop:- Configuration Reloading ::
What is the use of the Config Store if it can not accept the change and gracefully delegate to the required components. Reloading configuration from Kubernetes configmap without restarting the application properties is another great feature of this project. There can be many business scenarios where you need and this feature can be used. For example :-
- Dynamic Log level setting
- Configurable timeout values
- External host configurations, etc.
The examples can continue and is completely dependent on the business use case. Let’s see how this can be achieved. In the Item-Service ConfigMap I have a dummy url set as gateway for an external host.
These config property is read by a bean inside the application. We have @ConfigurationProperties to bind the ConfigMap properties.
To demonstrate the reload activity, we have set a scheduler that read the bean every 5 seconds and prints the configuration.
Following screenshot shows the scheduler is printing the message before and after the change
— Before Change
— After change
You can find more information related to config watcher on my other article [https://amit4aws.medium.com/spring-cloud-kubernetes-configwatcher-7a206fe269e3]
At this point we have structured our components with a persistence mechanism and have successfully externalized our configurations via config watcher.
