Client-Side Load-Balancing with Spring Cloud LoadBalancer

This guide walks you through the process of creating load-balanced microservices.

What You Will Build

You will build a microservice application that uses Spring Cloud LoadBalancer to provide client-side load-balancing in calls to another microservice.

What You Will Need

About 15 minutes
A favorite text editor or IDE
JDK 1.8 or later
Gradle 6+ or Maven 3.5+
You can also import the code straight into your IDE:
Spring Tool Suite (STS) or IntelliJ IDEA

Create a Root Project

This guide walks through building two projects, one of which is a dependency to the other. Consequently, you need to create two child projects under a root project. First, create the build configuration at the top level. For Maven, you need a pom.xml with <modules> that list the subdirectories:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.springframework</groupId>
    <artifactId>gs-spring-cloud-loadbalancer</artifactId>
    <version>0.1.0</version>
    <packaging>pom</packaging>

    <modules>
      <module>say-hello</module>
      <module>user</module>
    </modules>
</project>

For Gradle, you need want a settings.gradle that includes the same directories:

rootProject.name = 'gs-spring-cloud-loadbalancer'

include 'say-hello'
include 'user'

Optionally, you can include an empty build.gradle (to help IDEs identify the root directory).

Create the Directory Structure

In the directory that you want to be your root directory, create the following subdirectory structure (for example, with mkdir say-hello user on *nix systems):

└── say-hello
└── user

In the root of the project, you need to set up a build system, and this guide shows you how to use Maven or Gradle.

Starting with Spring Initializr

If you use Maven for the Say Hello project, visit the Spring Initializr to generate a new project with the required dependency (Spring Web).

The following listing shows the pom.xml file that is created when you choose Maven:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>2.4.4</version>
		<relativePath/> <!-- lookup parent from repository -->
	</parent>
	<groupId>com.example</groupId>
	<artifactId>spring-cloud-loadbalancer-say-hello</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<name>spring-cloud-loadbalancer-say-hello</name>
	<description>Demo project for Spring Boot</description>
	<properties>
		<spring-boot.repackage.skip>true</spring-boot.repackage.skip>
		<java.version>1.8</java.version>
	</properties>
	<dependencies>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
		</dependency>

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
			<scope>test</scope>
		</dependency>
	</dependencies>

	<build>
		<plugins>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
			</plugin>
		</plugins>
	</build>

</project>

If you use Gradle for the Say Hello project, visit the Spring Initializr to generate a new project with the required dependency (Spring Web).

The following listing shows the build.gradle file that is created when you choose Gradle:

plugins {
	id 'org.springframework.boot' version '2.4.4'
	id 'io.spring.dependency-management' version '1.0.11.RELEASE'
	id 'java'
}

group = 'com.example'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = '1.8'

repositories {
	mavenCentral()
}

dependencies {
	implementation 'org.springframework.boot:spring-boot-starter-web'
	testImplementation 'org.springframework.boot:spring-boot-starter-test'
}

test {
	useJUnitPlatform()
}

bootJar {
	enabled = false
}

If you use Maven for the User project, visit the Spring Initializr to generate a new project with the required dependencies (Cloud Loadbalancer and Spring Reactive Web).

The following listing shows the pom.xml file that is created when you choose Maven:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>2.4.4</version>
		<relativePath/> <!-- lookup parent from repository -->
	</parent>
	<groupId>com.example</groupId>
	<artifactId>spring-cloud-loadbalancer-user</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<name>spring-cloud-loadbalancer-user</name>
	<description>Demo project for Spring Boot</description>
	<properties>
		<spring-boot.repackage.skip>true</spring-boot.repackage.skip>
		<java.version>1.8</java.version>
		<spring-cloud.version>2020.0.0</spring-cloud.version>
	</properties>
	<dependencies>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-webflux</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.cloud</groupId>
			<artifactId>spring-cloud-starter-loadbalancer</artifactId>
		</dependency>

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
			<scope>test</scope>
		</dependency>
		<dependency>
			<groupId>io.projectreactor</groupId>
			<artifactId>reactor-test</artifactId>
			<scope>test</scope>
		</dependency>
	</dependencies>
	<dependencyManagement>
		<dependencies>
			<dependency>
				<groupId>org.springframework.cloud</groupId>
				<artifactId>spring-cloud-dependencies</artifactId>
				<version>${spring-cloud.version}</version>
				<type>pom</type>
				<scope>import</scope>
			</dependency>
		</dependencies>
	</dependencyManagement>

	<build>
		<plugins>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
			</plugin>
		</plugins>
	</build>
	<repositories>
		<repository>
			<id>spring-milestones</id>
			<name>Spring Milestones</name>
			<url>https://repo.spring.io/milestone</url>
		</repository>
	</repositories>

</project>

If you use Gradle for the User project, visit the Spring Initializr to generate a new project with the required dependencies (Cloud Loadbalancer and Spring Reactive Web).

The following listing shows the build.gradle file that is created when you choose Gradle:

plugins {
	id 'org.springframework.boot' version '2.4.4'
	id 'io.spring.dependency-management' version '1.0.11.RELEASE'
	id 'java'
}

group = 'com.example'
version = '0.0.1-SNAPSHOT'
sourceCompatibility = '1.8'

repositories {
	mavenCentral()
	maven { url 'https://repo.spring.io/milestone' }
}

ext {
	set('springCloudVersion', "2020.0.0")
}

dependencies {
	implementation 'org.springframework.boot:spring-boot-starter-webflux'
	implementation 'org.springframework.cloud:spring-cloud-starter-loadbalancer'
	testImplementation 'org.springframework.boot:spring-boot-starter-test'
	testImplementation 'io.projectreactor:reactor-test'
}

dependencyManagement {
	imports {
		mavenBom "org.springframework.cloud:spring-cloud-dependencies:${springCloudVersion}"
	}
}

test {
	useJUnitPlatform()
}

bootJar {
	enabled = false
}

Manual Initialization (optional)

If you want to initialize the project manually rather than use the links shown earlier, follow the steps given below:

Navigate to https://start.spring.io. This service pulls in all the dependencies you need for an application and does most of the setup for you.
Choose either Gradle or Maven and the language you want to use. This guide assumes that you chose Java.
Click Dependencies and select Spring Web (for the Say Hello project) or Cloud Loadbalancer and Spring Reactive Web (for the User project).
Click Generate.
Download the resulting ZIP file, which is an archive of a web application that is configured with your choices.

If your IDE has the Spring Initializr integration, you can complete this process from your IDE.

Implement the "Say Hello" service

Our “server” service is called Say Hello. It returns a random greeting (picked out of a static list of three) from an endpoint that is accessible at /greeting.

In src/main/java/hello, create the file SayHelloApplication.java.

The following listing shows the contents of say-hello/src/main/java/hello/SayHelloApplication.java:

package hello;

import java.util.Arrays;
import java.util.List;
import java.util.Random;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@SpringBootApplication
public class SayHelloApplication {

  private static Logger log = LoggerFactory.getLogger(SayHelloApplication.class);

  public static void main(String[] args) {
    SpringApplication.run(SayHelloApplication.class, args);
  }

  @GetMapping("/greeting")
  public String greet() {
  log.info("Access /greeting");

  List<String> greetings = Arrays.asList("Hi there", "Greetings", "Salutations");
  Random rand = new Random();

  int randomNum = rand.nextInt(greetings.size());
  return greetings.get(randomNum);
  }

  @GetMapping("/")
  public String home() {
  log.info("Access /");
  return "Hi!";
  }
}

It is a simple @RestController, where we have one @RequestMapping method for the /greeting and another for the root path /.

We are going to run multiple instances of this application locally alongside a client service application. To get started:

Create a src/main/resources directory.
Create a application.yml file within the directory.
In that file, set a default value for server.port.

(We will instruct the other instances of the application to run on other ports so that none of the Say Hello instances conflict with the client when we get that running). While we are in this file, we can set the spring.application.name for our service too.

The following listing shows the contents of say-hello/src/main/resources/application.yml:

spring:
  application:
    name: say-hello

server:
  port: 8090

Access from a Client Service

Our users see the User application. It makes a call to the Say Hello application to get a greeting and then sends that greeting to our user when the user visits the endpoints at /hi and /hello.

In the User application directory, under src/main/java/hello, add the UserApplication.java file:

The following listing shows the contents of user/src/main/java/hello/UserApplication.java

package hello;

import reactor.core.publisher.Mono;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.client.loadbalancer.reactive.ReactorLoadBalancerExchangeFilterFunction;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.reactive.function.client.WebClient;

/**
 * @author Olga Maciaszek-Sharma
 */
@SpringBootApplication
@RestController
public class UserApplication {

  private final WebClient.Builder loadBalancedWebClientBuilder;
  private final ReactorLoadBalancerExchangeFilterFunction lbFunction;

  public UserApplication(WebClient.Builder webClientBuilder,
      ReactorLoadBalancerExchangeFilterFunction lbFunction) {
    this.loadBalancedWebClientBuilder = webClientBuilder;
    this.lbFunction = lbFunction;
  }

  public static void main(String[] args) {
    SpringApplication.run(UserApplication.class, args);
  }

  @RequestMapping("/hi")
  public Mono<String> hi(@RequestParam(value = "name", defaultValue = "Mary") String name) {
    return loadBalancedWebClientBuilder.build().get().uri("http://say-hello/greeting")
        .retrieve().bodyToMono(String.class)
        .map(greeting -> String.format("%s, %s!", greeting, name));
  }

  @RequestMapping("/hello")
  public Mono<String> hello(@RequestParam(value = "name", defaultValue = "John") String name) {
    return WebClient.builder()
        .filter(lbFunction)
        .build().get().uri("http://say-hello/greeting")
        .retrieve().bodyToMono(String.class)
        .map(greeting -> String.format("%s, %s!", greeting, name));
  }
}

We also need a @Configuration class where we set up a load-balanced WebClient.Builder instance:

The following listing shows the contents of user/src/main/java/hello/WebClientConfig.java:

package hello;

import org.springframework.cloud.client.loadbalancer.LoadBalanced;
import org.springframework.cloud.loadbalancer.annotation.LoadBalancerClient;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.reactive.function.client.WebClient;

@Configuration
@LoadBalancerClient(name = "say-hello", configuration = SayHelloConfiguration.class)
public class WebClientConfig {

  @LoadBalanced
  @Bean
  WebClient.Builder webClientBuilder() {
    return WebClient.builder();
  }

}

The configuration provides a @LoadBalanced WebClient.Builder instance, which we use when someone hits the hi endpoint of UserApplication.java. Once the hi endpoint is hit, we use this builder to create a WebClient instance, which makes an HTTP GET request to the Say Hello service’s URL and gives us the result as a String.

In UserApplication.java, we have also added a /hello endpoint that does the same action. However, rather than use the @LoadBalanced annotation, we use an @Autowired load-balancer exchange filter function (lbFunction), which we pass by using the filter() method to a WebClient instance that we programmatically build.

Even though we set up the load-balanced WebClient instance slightly differently for the two endpoints, the end behavior for both is exactly the same. Spring Cloud LoadBalancer is used to select an appropriate instance of the Say Hello service.

Add the spring.application.name and server.port properties to src/main/resources/application.properties or src/main/resources/application.yml:

The following listing shows the contents of user/src/main/resources/application.yml

spring:
  application:
    name: user

server:
  port: 8888

Loadbalance Across Server Instances

Now we can access /hi or hello on the User service and see a friendly greeting:

$ curl http://localhost:8888/hi
Greetings, Mary!

$ curl http://localhost:8888/hi?name=Orontes
Salutations, Orontes!

In WebClientConfig.java, we pass a custom configuration for the LoadBalancer by using the @LoadBalancerClient annotation:

@LoadBalancerClient(name = "say-hello", configuration = SayHelloConfiguration.class)

This means that, whenever a service named say-hello is contacted, instead of running with the default setup, Spring Cloud LoadBalancer uses the configuration provided in SayHelloConfiguration.java.

The following listing shows the contents of user/src/main/java/hello/SayHelloConfiguration.java:

package hello;

import java.util.Arrays;
import java.util.List;

import reactor.core.publisher.Flux;

import org.springframework.cloud.client.DefaultServiceInstance;
import org.springframework.cloud.client.ServiceInstance;
import org.springframework.cloud.loadbalancer.core.ServiceInstanceListSupplier;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;

/**
 * @author Olga Maciaszek-Sharma
 */
@Configuration
public class SayHelloConfiguration {

  @Bean
  @Primary
  ServiceInstanceListSupplier serviceInstanceListSupplier() {
    return new DemoServiceInstanceListSuppler("say-hello");
  }

}

class DemoServiceInstanceListSuppler implements ServiceInstanceListSupplier {

  private final String serviceId;

  DemoServiceInstanceListSuppler(String serviceId) {
    this.serviceId = serviceId;
  }

  @Override
  public String getServiceId() {
    return serviceId;
  }

  @Override
  public Flux<List<ServiceInstance>> get() {
    return Flux.just(Arrays
        .asList(new DefaultServiceInstance(serviceId + "1", serviceId, "localhost", 8090, false),
            new DefaultServiceInstance(serviceId + "2", serviceId, "localhost", 9092, false),
            new DefaultServiceInstance(serviceId + "3", serviceId, "localhost", 9999, false)));
  }
}

In that class, we provide a custom ServiceInstanceListSupplier with three hard-coded instances that Spring Cloud LoadBalancer chooses from while making the calls to the Say Hello service.

This step has been added to explain how you can pass your own custom configuration to the Spring Cloud LoadBalancer. However, you need not use the @LoadBalancerClient annotation and create your own configuration for the LoadBalancer. The most typical way is to use Spring Cloud LoadBalancer with service discovery. If you have any DiscoveryClient on your classpath, the default Spring Cloud LoadBalancer configuration uses it to check for service instances. As a result, you only choose from instances that are up and running. You can learn how to use ServiceDiscovery with this guide.

We also add an application.yml file with default server.port and spring.application.name.

The following listing shows the contents of user/src/main/resources/application.yml:

spring:
  application:
    name: user

server:
  port: 8888

Testing the Loadbalancer

The following listing shows how to run the Say Hello service with Gradle:

$ ./gradlew bootRun

The following listing shows how to run the Say Hello service with Maven:

$ mvn spring-boot:run

You can run other instances on ports 9092 and 9999 To do so with Gradle, run the following command:

$ SERVER_PORT=9092 ./gradlew bootRun

To do so with Maven, run the following command:

$ SERVER_PORT=9999 mvn spring-boot:run

Then you can start the User service. To do so, access localhost:8888/hi and watch the Say Hello service instances.

Your requests to the User service should result in calls to Say Hello being spread across the running instances in round-robin fashion:

2016-03-09 21:15:28.915  INFO 90046 --- [nio-8090-exec-7] hello.SayHelloApplication                : Access /greeting

Summary

Congratulations! You have just developed a Spring Loadbalancer application!

Want to write a new guide or contribute to an existing one? Check out our contribution guidelines.

All guides are released with an ASLv2 license for the code, and an Attribution, NoDerivatives creative commons license for the writing.