On behalf of the Spring XD team, I am very pleased to announce that the Spring XD 1.1 Release Candidate is now available for download.

The 1.1 RC1 release includes several new features as well as bug fixes.

One theme in the 1.1 release is around Stream processing. The 1.1 M2 version introduced support for Reactor’s Stream API in processing modules. The 1.1 RC1 release adds support for stream processing using RxJava’s Observable API and Spark Streaming alongside the existing Spark job support. By providing a range of options, you can pick the functional programming model that best suits the task at hand.

Spark Streaming integration features include:

• XD Processor modules that act as drivers for workloads on the Spark Cluster.
• An XD processor’s input DStream is generated from data received on the XD Message Bus, and the RDDs from its output DStream are sent to the bus. This allows you to immediately take advantage of the many Source modules to drive computation in Spark and direct the output to XD Sinks. This removes a great amount of complex code that you would need to write otherwise, such as mapping Kafka partitions to multiple DStreams.
• The Spark Streaming driver process is automatically restarted if it dies due to the inherent module redeployment features of XD.
• Simplify development and testing of Spark applications by decoupling your Spark computation code from code to setup input and output data. This allows creating integration tests using a test message bus to send specific data to the Spark module’s input channel and assert on the results contained in the output channel.

RxJava integration features include:

• Development of processor modules using RxJava’s Observable API This allows you to use a functional programming model to process event streams. Combined with XD’s support for Data Partitioning on the Message Bus, you can create very powerful distributed stream processing solutions.
• Support for mapping input messages to one or multiple Observerble Stream instances, for example allowing a Stream per Kafka partition.
• Refer to the rxjava-moving-average project for an example of using RxJava’s Observable API.

Other features include:

• Container Group Management when running XD on YARN. You can now create groups of XD Admin and XD Container instances on YARN. Groups can specify the number of initial member instances, ramp up or down the number of instances at runtime, and specify specific machines or racks where they will be deployed. If any YARN containers fail, they will automatically restarted to maintain the number of instances specified for the group.
• Refactored Kafka Based Message Bus and Kafka Source based on the new Spring Integration Kafka client library. For the Kafka Source, this translates into new options, such as specifying the partitions that a source can listen to, or the starting offsets for replayability.
• Gemfire source and sink support for multiple locators.
• Improved tooling and updated examples for custom module development.

Feedback is very important, so please keep in touch with questions and comments via

• StackOverflow spring-xd tag
• Spring JIRA or GitHub Issues