The release 2.3.0 delivers a lot of enhancements and generic compatibility changes for Spring Boot 2.2.x and Spring Cloud Hoxton.

Continuous Deployment For Tasks

As task applications evolve faster to keep up with business needs, the ability for new versions to be consumed via Data Flow in an automated way is needed. While Data Flow has supported the ability to register multiple versions of a task application in previous iterations, the ability to run them in a practical way by re-hydrating command line arguments, deployment properties, and application properties used in previous executions has been missing. In this version, the storage of those values in a manifest and the ability to both retrieve them to determine if an application needs to be upgraded and apply them to the new execution allows for developers to create continuous deployment flows for their task applications. All these capabilities are readily available through RESTful APIs, as well, so the overall CI/CD workflow for Tasks can be automated.

Scheduler improvements

Scheduling has been updated to support the Task’s CI/CD features. Thus when the Kubernetes or Cloud Foundry Task-scheduler launches the application after the user updates the Task application to a new version, the next execution in Spring Cloud Data Flow will take advantage of these updates automatically.

Monitoring improvements

In this release, we have revised the monitoring architecture to drive towards a consistent experience in Local, Kubernetes, and Cloud Foundry.

• Using Prometheus RSocket Proxy as a default approach for Prometheus-based monitoring of the short lived Tasks, as well as long-lived streaming applications, and across all supported platforms.
• Native monitoring of Spring Cloud Tasks and Spring Cloud Batch, complements to the existing monitoring support for streaming applications through Spring Cloud Streams.
• We have a few monitoring-samples. You will find instructions for building custom Stream and Task applications integrated with the Data Flow Monitoring Architecture:
  • stream-apps - how to enable monitoring for custom built source, processor and sink apps.
  • task-apps - how to enable monitoring for custom built task apps.
• Allow using the monitoring architecture to implement elastic, auto-scaling adapters for stream pipelines.

Kubernetes deployer improvements

The following new capabilities are readily available as deployment properties in Kubrentes for both streaming and batch data pipelines.

• Support for Node Affinity, Pod Affinity and Anti-Affinity
• Ability to add multiple ports to Service objects
• Allow customization of the container image used in StatefulSet deployments
• Implementation of Scaling API
• Support for custom init containers

Helm chart improvements

• Ability to enable monitoring support using Prometheus and Grafana
• Try out the 2.3 GA compatible version of the Helm Chart and let us know what you think!

Scaling API

The addition of new scaling API is available to quickly alter the number of application instances without redeploying a whole stream with updates to the deployment properties. The Scaling API is agnostic to the target platform and can be used seamlessly with K8s, CF and Local.

Import/Export Utility

We also made it easier to work with multiple environments by adding Import/Export Streams feature which provides easy moving streams across different environments, e.g. dev, test, prod.

Security

Our journey to fully move into next generation Spring Security OAuth2 support is almost complete and we expect to finalize it in next releases. There is a blog post Next Generation OAuth 2.0 Support with Spring Security which outlined where things are going in a Spring world.

Developer Surveys

We have released one major release (2.0), three minor releases (2.1, 2.2, and 2.3), and several maintenance releases in 2019! Likewise, Spring Cloud Stream, Spring Cloud Task, Deployers, Skipper, Apps, and remaining others in the SCDF ecosystem independently evolved in isolation, as well.

Thank you, everyone, for your support, contributions, and participation!

As we are inching towards the New Year’s, we wanted to reach out to the community to learn about your interests and feedback. Please take these super quick 1-page surveys and let us know.

Spring Cloud Data Flow Spring Cloud Stream Spring Cloud Task

Stay in touch...

As always, we welcome feedback and contributions, so please reach out to us on Stackoverflow or GitHub or via Gitter.