Spring Blog

On behalf of the Spring XD team, I am very pleased to announce the general availability of Spring XD 1.0! You can download the zip distribution. You can also install on OSX using homebrew and on RHEL/CentOs using yum.

Spring XD’s goal is to be your one stop shop for developing and deploying Big Data Applications. Such applications require a wide range of technologies to address different use-cases while interoperating as a cohesive process. The steps in this process include:

• Data collection
• Real-time streaming and analytics
• Data cleansing
• Batch processing (both on and off Hadoop)
• Machine learning and exploratory data analysis
• Visualization and Reporting
• Closed loop analytics between real-time and batch processing

Spring XD brings together many of these steps into a single unified runtime platform so you can address as many use-cases as possible. You don’t have to piece together a large number of individual projects, each with its own configuration and programming model. Instead, with Spring XD, you can quickly get started developing an end-to-end solution with a simple but powerful DSL and scale it out.

Spring XD provides:

• A distributed, fault tolerant and scalable runtime
• A unified platform for Stream and Batch Processing supporting
  • Popular sources, processors, and sinks for stream processing
  • Stream processing using NoSQL Analytics
  • Hadoop Batch workflow orchestration
  • Off-Hadoop Batch Jobs
  • Closed loop machine learning from Batch to Real-time scoring via JPMML
• Runtime portability: easy to setup, operate and deploy on VM/on-metal cluster, YARN, or EC2
• Hadoop distribution portability: certified against six Hadoop distributions
• Enterprise readiness as it is built on proven Spring foundation projects such as Spring Integration, Spring Batch, and Spring Data
• Ease of use
  • DSL to create streams and jobs using many pre-built components
  • Interactive Shell with tab completion
  • Admin UI and REST API
  • Easy to develop and test custom modules as well as extend the runtime.
  • Custom modules can easily incorporate other Spring projects, such as those in Spring Data, e.g. Spring Data REST for creating dashboards

There are several resources available to help you get started using Spring XD.

• InfoQ article
• Spring XD Meetup Recording
• Getting Started Guide
• Project Page
• Reference Documentation

There are many samples available in our samples repository Here are a few that show the range of functionality available
* Twitter Streaming Dashboard
* Batch Analysis of Tweet hashtags
* Scoring in real-time with JPMML

This was a great team effort, with over a dozen active contributors spread around the world working for 68 weeks, divided into 32 sprints, resolving 1,650 issues in 2000 commits, 1,000 tests, and 270 pages of documentation. The 51,000 HipChat messages helped keep the team in constant contact and searching for new emoticons and fun images

Thanks for all the feedback from early adopters. Feedback is very important, so please get in touch with questions and comments via
* StackOverflow spring-xd tag
* Spring JIRA or GitHub Issues

SpringOne 2GX 2014 is around the corner

Book your place at SpringOne in Dallas, TX for Sept 8-11 soon. It’s simply the best opportunity to find out first hand all that’s going on and to provide direct feedback. There will be deep dive sessions on Spring XD along with general Big Data talks to provide an introduction to the landscape and challenges in developing Big Data applications.