Hear from the Spring team this January at SpringOne. >
close

Gary Russell

Gary Russell

Project Lead, Spring for Apache Kafka, AMQP

Pennsylvania, USA

Gary has been in software engineering, concentrating on Enterprise Integration, for nearly 40 years on various platforms, and in the Java space since the late '90s. He has been developing with the Spring Framework since 2004. He has been a committer on the Spring Integration and Spring AMQP projects for over 10 years; he started the Spring for Apache Kafka project in 2016. He leads both Spring AMQP and Spring for Apache Kafka and is a former lead of Spring Integration.
Blog Posts by Gary Russell

New Major Releases of Spring for Apache Kafka and Spring for RabbitMQ

I am pleased to announce that new major releases (3.0.0) for Spring for Apache Kafka and Spring for RabbitMQ are now available.

Common Changes

The 3.0.x versions of these projects have the following common changes

  • they depend on Java version 17 or later

  • they depend on Spring Framework 6.0.x

  • support for creating native GraalVM applications

  • support for micrometer observability and tracing

  • Bill of Materials POMs are now provided to help with dependency management

Spring for Apache Kafka

  • support for a global single EmbeddedKafkaBroker (across multiple test classes)

  • several enhancements for @RetryableTopic (non-blocking retries)

  • KafkaTemplate methods now return CompleteableFuture instead of ListenableFuture, which has been deprecated

For information about all changes in this release, see What’s New.

In addition, 2.9.3 and 2.8.11 maintenance releases are now available.

Read more...

Spring for Apache Kafka 2.9.0 Available

I am pleased to announce that Spring for Apache Kafka 2.9.0 is now available.

This version is not provided by Spring Boot dependency management, but it can be used with Boot 2.7 (or 2.6), as long as you override the Kafka dependencies to 3.2.0 as described in this appendix, especially if you are using the embedded Kafka broker.

Please see the release candidate announcement for notable changes in this release.

For information about all changes in this release, see What’s New.

Read more...

Spring for Apache Kafka 2.9 Release Candidate Available

I am pleased to announce that the release candidate Spring for Apache Kafka 2.9.0 is now available (2.9.0-RC1) in the Spring Milestone Repo.

This version is not provided by Spring Boot dependency management, but it can be used with Boot 2.7 (or 2.6), as long as you override the Kafka dependencies as described in this appendix, especially if you are using the embedded Kafka broker.

Notable Changes

  • This version uses the 3.2.0 kafka-clients version

  • Non-blocking retry bootstrapping is now more robust

  • New Error Handler Mode

By default, after an error, the DefaultErrorHandler performs seeks on the remaining records from the last poll and re-fetches them from the broker on the next poll. With high error rates and large max.poll.records, this can cause unnecessary strain on the network. For this reason, the error handler has a new property seekAfterError, when set to false, instead of seeking the records, the remaining records are retained in memory and the consumer paused for the next poll (or multiple polls if the error handler is configured to use the ContainerPausingBackOffHandler).

  • Pausing Containers

By default, when you pause() a container, it actually pauses when all the records from the previous poll have been processed. This version adds the pauseImmediate container property, which, when true, causes the container to pause after the current record is processed.

For information about all changes in this release, see What’s New.

Read more...

Security Report for Spring AMQP (Spring for RabbitMQ)

The recently released versions of Spring AMQP (2.4.0, 2.3.12, 2.2.20) contain a fix for CVE-2021-22095, which can cause a potential OutOfMemoryError for very large messages.

Spring Boot version 2.6.0 and 2.5.7 will bring in the 2.4.0 and 2.3.12 versions of spring-amqp respectively. Users of Boot 2.4.x should override the spring-amqp and spring-rabbit versions to 2.3.12; users of Spring Boot 2.3.x should override the versions to 2.2.20.

Read more...

Spring for Apache Kafka 2.8 is Now Available

I am pleased to announce that Spring for Apache Kafka 2.8.0 is now available.

Notable Changes

  • This version uses the 3.0.0 kafka-clients version

  • Out of order manual commits are now supported

  • The same container factory can now be used for both batch and record listeners

  • Separate record and batch error handlers have been replaced with common error handlers

  • KafkaTemplate can now be used to receive specific records from known partitions/offsets

For information about all changes in this release, see What’s New.

Read more...

Spring for Apache Kafka 2.7.0 Available

I am pleased to announce that Spring for Apache Kafka 2.7.0 is now available.

This release contains a significant enhancement, which is a community contribution. Failed deliveries can be forwarded to a series of topics for delayed redelivery.

It is best described with an example:

@RetryableTopic(attempts = "5", backoff = @Backoff(delay = 1000, multiplier = 2.0))
@KafkaListener(id = "sk270", topics = "sk270")
public void listen(String in, @Header(KafkaHeaders.RECEIVED_TOPIC) String topic) {
    LOG.info(in + " from " + topic);
    throw new RuntimeException("test");
}

@DltHandler
public void dlt(String in, @Header(KafkaHeaders.RECEIVED_TOPIC) String topic) {
    LOG.info(in + " from " + topic);
}
Read more...

Spring for Apache Kafka 2.5.0 Release Candidate

The 2.5.0.RC1 release candidate is now avaialable in the Spring milestone repo.

Update: 2.5.0.RELEASE was released on May 13th.

Highlights:

  • kafka-clients 2.5.0 (alignment of version numbers is coincidental).

  • Support for re-committing retryable offset commit exceptions for retained partitions when using cooperative rebalancing.

  • Support for the new "fetch-offset-request" procuder fencing (when brokers are 2.5 or higher), requiring fewer producers.

  • Support for static group membership.

  • More integration with Micrometer.

  • Optional Delivery Attempts header.

  • RecoveringBatchErrorHandler can commit a partial batch and replay from failed record in a batch (with cooperation of the listener); this is now the default for a batch listener.

  • Default error handler for record listener is now the SeekToCurrentErrorHandler.

  • Overridable producer properties in the KafkaTemplate allowing multiple templates to use the same producer factory.

  • Simple String serializer and deserializer are now provided.

  • More flexibility to determin the type to create in the JsonDeserializer.

Read more...

Spring for Apache Kafka 2.4 is Available

To support last week’s Apache Kafka 2.4.0 release, I am pleased to announce that the Spring for Apache Kafka 2.4 - 2.4.0.RELEASE - is available in maven central.

This version is essentially functionally equivalent to 2.3.x, but is compiled against the 2.4.0 kafka-clients and supports the new incremental rebalancing protocol.

The 2.4.0 kafka-clients are not binary compatible with Spring for Apache Kafka 2.3 so if you wish to use the 2.4.0 clients, you must upgrade to this version. See the appendix in the reference manual for how to override the jar versions, especially if you are using Spring Boot for dependency management and/or you are using the test embedded Kafka broker.

Read more...

Spring for Apache Kafka 2.4 Release Candidate

Hot on the heels of the recent Apache Kafka 2.4.0 release, I am pleased to announce the release candidate for Spring for Apache Kafka 2.4 - 2.4.0.RC1 - is available in the Spring milestone repository.

This version is essentially functionally equivalent to 2.3.x, but is compiled against the 2.4.0 kafka-clients and supports the new incremental rebalancing protocol.

The 2.4.0 kafka clients are not binary compatible with Spring for Apache Kafka 2.3 so if you wish to use the 2.4.0 clients, you must upgrade to this version. See the appendix in the reference manual for how to override the jar versions, especially if you are using Spring Boot and/or the test embedded kafka broker.

Read more...