On behalf of the Spring Cloud Data Flow team, I am pleased to announce the 1.0.0.M2 release of Spring Cloud Data Flow for Apache YARN.

Spring Cloud Data Flow for Apache YARN allows one to use all the goodness of Spring Cloud Data Flow (like the Shell, UI and Flo) while targeting Apache YARN as a backend. Stream components are deployed as individual apps in Apache YARN, leveraging the power of the platform to handle scaling and health monitoring.

This second milestone

• builds upon the 1.0.0.M1 release of Spring Cloud YARN Deployer.
• builds upon the 1.0.0.M3 release of Spring Cloud Data Flow.
• Adds Spring Cloud Task support to orchestrate short-lived tasks including Spring Batch jobs in a YARN cluster.
• Improved IP address discovery for internal coordination.
• Adds support to host, resolve and register OOTB apps from HDFS.
• Adds installable RPM bits.
• Adds Ambari plugin to provision SCDF on YARN.

This release supports running both streams and tasks on YARN. Data Flow YARN server can be either run against existing YARN cluster or automatically installed into Apache Ambari via specific Ambari integration plugin. See details from Reference Documentation.

With YARN it's easy to scale streams:

For a complete list of changes and improvements, please refer to 1.0.0.M2 release.