Centralize Configuration with Spring Cloud Config

Posted on September 9, 2022

In this post, we will go over the setup to centralize configuration using Spring Cloud Config for a distributed microservices architecture.

In a Microservice architecture where each service has its own configuration, it becomes very difficult to manage the configuration for each service, especially if it’s in different places. Spring Cloud Config provides a Config Server and Client side support for externalizing configuration. This is especially important in a microservices architecture where each service might have its own set of configurations.

In this article, we will set up a config server with Apache Kafka using Spring Cloud bus and then connect it with Config Clients built using spring. We will then use GitHub as the storage for our configuration and its webhook to call the monitor endpoint, for the config server to initiate the refresh workflow as shown below:

centralize configuration using Spring Cloud Config

Pre-Requisites

We will be referring to the setup done in this article, for creating a SpringBoot application with the ability to push images to a docker-compliant registry and helm charts for it to run on Kubernetes.

Config Server Setup

Create a spring boot application with the following dependencies

    implementation "org.springframework.boot:spring-boot-starter-actuator"
    implementation "org.springframework.cloud:spring-cloud-config-server"
    implementation "org.springframework.cloud:spring-cloud-config-monitor"
    implementation "org.springframework.cloud:spring-cloud-starter-stream-kafka"

Refer to the full Gradle file for more details.

Make these changes into application.properties

spring.cloud.config.server.git.uri=https://github.com/ribakored/spring-config-repo
spring.cloud.bus.enabled=true
spring.cloud.stream.kafka.binder.brokers=mstack-kafka-0.mstack-kafka-headless.default.svc.cluster.local
spring.cloud.stream.kafka.binder.auto-create-topics=true
spring.kafka.template.default-topic=springCloudBus

This setup connects the config server with the GitHub repository at https://github.com/ribakored/spring-config-repo to centralize configuration.

Config Client Setup

Create a Spring boot application using Spring Initializer with the following dependencies. 

    implementation "org.springframework.boot:spring-boot-starter-actuator"
    implementation "org.springframework.boot:spring-boot-starter-web"
    implementation "org.springframework.cloud:spring-cloud-starter-config"
    implementation "org.springframework.cloud:spring-cloud-starter-bootstrap"
    implementation "org.springframework.cloud:spring-cloud-starter-bus-kafka"

Refer to the full Gradle file for more details.

Add the following to the application and bootstrap property files

application.properties

spring.cloud.bus.id=configclient:dev
spring.cloud.bus.enabled=true
spring.cloud.stream.kafka.binder.brokers=mstack-kafka.default.svc.cluster.local

bootstrap.properties

spring.cloud.config.uri=http://configserver:8888
spring.application.name=configclient
spring.cloud.config.name=configclient
spring.profiles.active=dev

Helm Chart Setup

Next up, we will create a helm chart, which will implement the following scaled-down version of the architecture diagram above:

The helm chart is built with the Bitnami Kafka chart as a dependency. It creates a config client, config server, Kafka broker, and zookeeper pods. 

dependencies:
  - name: kafka
    version: 18.3.1
    repository: https://charts.bitnami.com/bitnami

The full helm chart can be found here.

Install the Helm chart

Now we will install the helm chart. Ensure that you have a flavor of k8s installed locally, see this post for more details.

Create images for both config client and config repo clients by running jibDocker task in the Gradle file. 

git clone https://github.com/ribakored/spring-kitchen-sink.git && cd spring-config-server-example/config-client-1
./gradlew -i jibDocker
cd ../config-server && ./gradlew -i jibDocker && cd ../

This will create and push images to the local repo. Then navigate to the helm directory and install the helm chart, ensuring that values.yaml has the correct tag names for both config server and config client applications. 

cd helm/configwithspringcloud && helm dependency update
cd ../ && helm install mstack configwithspringcloud/

Make sure you use mstack as the release name as I have used this as a prefix in bootstrap.properties. This part can be optimized further if need be.

After a few minutes, all the pods should be started and in a healthy state. There might be a few restarts as the config server and clients are waiting for Kafka brokers to start, but eventually all should be in a healthy state. 

$ kubectl get pods
NAME                           READY   STATUS    RESTARTS        AGE
configclient-8f7cf8bd8-2779x   1/1     Running   2 (86s ago)     3m40s
configserver-9b847bc78-jc2fd   1/1     Running   4 (67s ago)     3m40s
mstack-kafka-0                 1/1     Running   1 (2m17s ago)   3m40s
mstack-zookeeper-0             1/1     Running   0               3m40s

Call /monitor Endpoint

Next Up, Since the SpringBoot client application started before the config server was up, it will have default values from its application.properties

$ curl http://localhost/timeout
Timeout=null

Since its hard to create a Webhook in Github to make a post call on /monitor endpoint in a locally running Kubernetes cluster, we will call the post URL directly simulating what Github Webhook would actually do:

curl --location --request POST 'http://localhost:8888/monitor' \
--header 'X-Github-Event: push' \
--header 'Content-Type: application/json' \
--data-raw '{
  "commits": [
    {
      "modified": [
        "configclient-dev.properties"
      ]
    }
  ],
  "head_commit": {
    "modified": [
      "configclient-dev.properties"
    ]
  }
}'

After the monitor endpoint is called, you will see the following in logs within the client application:

2022-09-09 19:07:07.732  INFO 1 --- [container-0-C-1] o.s.cloud.bus.event.RefreshListener      : Received remote refresh request.
2022-09-09 19:07:07.836  INFO 1 --- [container-0-C-1] c.c.c.ConfigServicePropertySourceLocator : Fetching config from server at : http://configserver:8888
2022-09-09 19:07:08.176  INFO 1 --- [container-0-C-1] c.c.c.ConfigServicePropertySourceLocator : Located environment: name=configclient, profiles=[dev], label=null, version=d8a403c4b9c4f40298d163c7f519863593902679, state=null
2022-09-09 19:07:08.178  INFO 1 --- [container-0-C-1] b.c.PropertySourceBootstrapConfiguration : Located property source: [BootstrapPropertySource {name='bootstrapProperties-configClient'}, BootstrapPropertySource {name='bootstrapProperties-https://github.com/ribakored/spring-config-repo/configclient-dev.properties'}]
2022-09-09 19:07:08.195  INFO 1 --- [container-0-C-1] o.s.boot.SpringApplication               : The following 1 profile is active: "dev"
2022-09-09 19:07:08.223  INFO 1 --- [container-0-C-1] o.s.boot.SpringApplication               : Started application in 0.48 seconds (JVM running for 476.125)
2022-09-09 19:07:08.488  INFO 1 --- [container-0-C-1] o.s.cloud.bus.event.RefreshListener      : Keys refreshed [config.client.version, app.timeout]
2022-09-09 19:07:08.494  INFO 1 --- [container-0-C-1] o.s.c.s.binder.DefaultBinderFactory      : Retrieving cached binder: kafka
2022-09-09 19:07:08.532  INFO 1 --- [container-0-C-1] o.s.c.s.b.k.p.KafkaTopicProvisioner      : Using kafka topic for outbound: springCloudBus

Now if you hit the URL again, you will the value picked up from the config server file: 

$ curl http://localhost/timeout
Timeout=30

If using MiniKube instead of docker desktop you will need to get the service ip and replace it with localhost. Also dont forget to run minikube tunnel command in a separate terminal window.

Next Steps

In Addition to Apache Kafka (As Message Broker) and Github as Config storage following alternatives are also available depending on your use case

Kafka Alternatives

Following Messages systems can be used instead of Apache Kafka:

For further details refer to spring cloud stream documentation.

Config Storage Alternatives

Summary

To Summarize, to centralize configuration this article went through the setup of the most common use case for Spring Cloud Config server using Github as the Backend. You may still want to go through the various configuration options available to further tune the setup to your requirements. Here are some documents which can be referred to for more details: