Spring Cloud Data Flow 1.2 GA released

On behalf of the team, I am pleased to announce the general availability of Spring Cloud Data Flow 1.2 across a range of platforms

Here are the relevant links to documentation and getting started guides.

Highlights of the 1.2 release:

Composed Tasks

This release introduces Composed Tasks ! This feature provides the ability to orchestrate a flow of tasks as a cohesive unit-of-work. A complex ETL pipeline may include executions in sequence, parallel, conditional transitions, or a combination of all of the above. The composed task feature comes with DSL primitives and an interactive graphical interface to quickly build these type of topologies more easily. You can read more about it from the reference guide.

An ETL job, for example, may include multiple steps. Each step in the topology can be built as a finite short-lived Spring Cloud Task application. The orchestration of multiple tasks as steps can be easily defined with the help of the Data Flow Task DSL.

task create simple-etl --definition "extractDbToHDFS &&
      <analysisInSpark || enrichAndLoadHawq> &&
      <populateMgmtDashboard || runRegulatoryReport || loadAnalyticsStore>"

This will first run extractDbToHDFS and then run analysisInSpark and enrichAndLoadHawq in parallel, waiting for the both of them to complete before running the three remaining tasks in parallel and waiting for them to all complete before ending the job. The graphical representation of this topology looks like the following.

Visualization of Composed Tasks

Real-time Metrics and Monitoring

Real-time metrics are now part of the operational view of deployed streams. The applications that are part of a stream publish metrics contained in their Spring Boot /metrics actuator endpoint. This includes send and receive messages rates. A new server, the Spring Cloud Data Flow Metrics Collector, collects these metrics and calculates aggregate message rates. The Data Flow server queries the Metrics Collector to support showing message rates in the UI and in the shell. For more details about the architecture, refer to the Monitoring Deployed Applications Section in the reference guide.

The screenshot below shows the aggregate message rates for a time | log stream with three instances of the time and log applications. Each dot below the main application box shows the message rates for each individual application along with a guid value that can be used to identify the application on the platform where they are running.

Visualization of Input and Output Rates in Flo

The Runtime tab, shown below, also had improvements to show message rates and any other metrics exposed by the platform. For the script savvy users, the shell experience also includes these details via the runtime apps command.

Runtime Apps UI

Companion Artifact

The companion artifact support introduced in 1.2 M3 has had some improvements. The bulk registration workflow now eagerly resolves and downloads the metadata artifacts for all the out-of-the-box applications. This comes handy in the Shell or UI when reviewing the supported properties for each application.

OAUTH Improvements

This change will provide an additional option for REST-API users. Instead of providing a username:password combination via BasicAuth, users will now have the ability to retrieve an OAuth2 Access token from their OAuth2 provider directly and then provide the Access Token in the HTTP header, when invoking RESTful calls against a secured Spring Cloud Data Flow setup.

Role based access

Add role-based access control to define who has access to create, deploy, destroy, or view streams/tasks. This works seamlessly in coordination with the supported authentication methods.

Bug reporting

A new REST endpoint and About page in the Dashboard to collect server implementation details to the clipboard for use in bug reporting.

Spring Cloud Stream App Starters - Bacon.RELEASE

The Stream App Starters Bacon.RELEASE is now generally available which provides you a range of sources, processors, and sinks to get started creating stream. All the out-of-the-box stream applications build upon Spring Cloud Stream Chelsea.RELEASE and Spring Cloud Dalston.RELEASE foundation. There were several enhancements and bug-fixes to the existing applications and this release-train also brings new applications such as MongoDB-sink, Aggregator-processor, Header-Enricher-processor, and PGCopy-sink.

For convenience, we have generated the links that includes the latest coordinates for docker and maven artifacts.

Spring Cloud Task App Starters - Belmont.RELEASE

The Task App Starters Belmont.RELEASE release is now complete. To support Composed Task feature in Spring Cloud Data Flow, we have added a new out-of-the-box application named Composed Task Runner. This is a task that executes others tasks in a directed graph as specified by a DSL that is passed in via the --graph command line argument.

The Belmont.RELEASE builds upon Spring Cloud Task 1.2 RELEASE and Spring Cloud Dalston.RELEASE foundation.

For convenience, we have generated the links that includes the latest coordinates for docker and maven artifacts.

What’s Next?

An immediate goal is adding more automated integration tests and to expose this as an additional user facing feature. You can track that work here.

Beyond the 1.2.x line, we are going to start planning for the 2.0 version. Some general themes are support for deploying individual applications and keeping track of application deployment properties and metadata such as the application version. This functionality would build up into supporting a rich Continuous Delivery theme at the application level that also extends to "editing" streams at runtime. In addition, we are also looking into supporting functions, either "in-line" as Java code or compiled java.util.Function s to be a first class programming model for data processing a stream.

Feedback is important. Please reach out to us in StackOverflow and GitHub for questions and feature requests. We also welcome contributions! Any help improving the Spring Cloud Data Flow ecosystem is appreciated.

Read more

Spring Batch 4.0.0.M2 is now available

We are pleased to announce that Spring Batch 4.0.0.M2 is now available via Github and the Pivotal download repository. This release represents the second milestone for the Spring Batch 4.0 release. Many thanks to all that contributed to this release.

What’s new?

This milestone continues the work laid out in Spring Batch 4.0.0.M1. Specific updates include:

  • Dependency updates
  • Continued java configuration improvements

Dependency updates

Spring Batch 4 will be the version included in Spring Boot 2 so we’re taking this time to syncronize our dependencies. With that, this release supports upgrades to the latest milestones of the Spring Data Kay release train and Spring Integration 5.

Read more

Spring REST Docs 1.2.1.RELEASE

It’s my pleasure to announce that Spring REST Docs 1.2.1.RELEASE is available from Maven Central, JCenter, and our release repository. My thanks to everyone who contributed to this release by reporting bugs and opening pull requests.

This maintenance release includes a handful of bug fixes and documentation improvements. It is a recommend upgrade for all Spring REST Docs users.

GitHub | Issues | Documentation | Twitter | Gitter | Stack Overflow

Read more

Spring Security 5.0.0 M1

On behalf of the community, I’m pleased to announce the release of Spring Security 5.0.0 M1. This release includes bug fixes, new features, and is based off of Spring Framework 5.0.0 RC1. The highlights of the release include:

Initial Support for Reactive Web Applications

Following one of the primary themes of Spring Framework 5.0, Spring Security 5.0 will add support for Reactive applications by building on top of Spring’s reactive support. The first milestone focused on getting primary infrastructure in place.

Read more

Spring Session 2.0.0 M1 Released

On behalf of the community I’m pleased to announce the release of Spring Session 2.0.0.M1. This release is focused primarily on ensuring compatability with Spring Framework 5 which is the minimum Spring version required.

Supported Data Stores

We have also removed some of the Spring Session implementations from the main repository. The goal is to allow the core Spring Session team to focus on delivering new features rather than needing to know the ins and outs of every data store. This will allow development of other modules to be done without the overhead of reviews from the Spring Session team.

Read more

Pivotal Announces Spring Curriculum & Certification Changes


It has been almost 10 years now since I taught my first Core Spring class. At that time almost everything was XML and configuring JPA or Spring Security, for example, could be a lot of hard work. Spring has matured a great deal in the meantime with component-scanning, Java Configuration and Spring Boot making it much more fun to use. And the number of Spring Projects has increased considerably.

Pivotal Training continues to enhance our Spring curriculum and introduce new courses - including Spring Boot Developer and Spring Cloud Services. I have provided an overview of these courses and some of the newest features below. Most importantly, how you obtain Certification has changed.

Read more

Spring Data Release Train Kay M3 Released

On behalf of the Spring Data team, I’d like to announce the availability of the third milestone of the release train Kay.

The most significant change is an improved naming scheme for CrudRepository. We extracted the store-specific CustomConversions to Spring Data Commons and introduced bi-directional type-based converters to maintain a single API for common conversion code. You can now define converters using lambdas. You need to adopt your code to the renamed methods. MongoDB ships with a renamed annotation for reactive tailable cursors, we renamed @InfiniteStream to @Tailable. Users of Spring Data for Apache Cassandra can now upgrade to a higher Guava version as we upgraded to Datastax driver 3.2.0.

Read more

This Week in Spring - May 8th, 2017

Hi Spring fans! This week I’m in Austin, Texas for OSCON; London, UK, for Devoxx UK; and Nice, France for RivieraDev! If you’re around, I’d love to talk to you so don’t hesitate to say hi!

Read more

Spring AMQP 2.0.0 Milestone 4 is Available

I am pleased to announce that the fourth milestone of Spring AMQP 2.0 2.0.0.M4 is now available in the Spring milestone repository.

Features since the last milestone include…​

  • The new DirectMessageListenerContainer now supports configuring the number of messages to process between acks.

  • You can now specify container concurrency on the @RabbitListener annotation.

The release candidate is expected to be released in early June with the GA shortly after the Spring Framework 5.0 GA release.

For a complete list of changes in 2.0, see What’s New in the reference manual; also see the release notes for 2.0.0.M1, 2.0.0.M2, 2.0.0.M3, as well as 2.0.0.M4.

Read more

Spring Cloud Camden SR7 Released

On behalf of the team, I am pleased to announce that Service Release 7 of the Spring Cloud Camden Release Train is available today. The release can be found in our Spring Release repository and Maven Central. The documentation can be found here. The release notes can be found here.

The following modules were updated as part of Camden.SR7:

  • Spring Cloud Commons 1.1.9.RELEASE
  • Spring Cloud Netflix 1.2.7.RELEASE
  • Spring Cloud AWS 1.1.4.RELEASE
  • Spring Cloud Consul 1.1.4.RELEASE Dependency Fixes
  • Spring Cloud Security 1.1.4.RELEASE Documentation Fixes
  • Spring Cloud Sleuth 1.1.4.RELEASE
  • Spring Cloud Stream Brooklyn.SR3 fixes issues with Spring Cloud Bus, Hystrix, and Turbine. See this issue for more information.
  • Spring Cloud Contract 1.0.5.RELEASE
Read more