close

Introducing Spring YARN getting started guides

Now that Spring for Apache Hadoop version 2.0 is GA I’d like use this opportunity to introduce our new Spring IO getting started guides for builing Hadoop YARN applications.

We have a generic guides how to work with Spring YARN using Gradle or Maven. Our guides are designed to work with both build systems.
* Building Spring YARN Projects with Gradle
* Building Spring YARN Projects with Maven

These two guides are demonstrating a familiar ‘hello world’ application type having a very minimalistic code base. Behaviour of these applications are identical having a difference in a project structure. While most of our guides are based on multi-project structures, building an application using Spring YARN surely is not limited to that. The second one demonstrates how an application can be created using a single project.
* Simple YARN Application
* Simple Single Project YARN Application

Spring YARN contains first class testing support for YARN applications. Focus of this guide is to show how a simple application can be tested using Hadoop’s mini clusters as cluster replacement in your unit tests. You don’t need to have a real running Hadoop instance.
* Testing YARN Application

The last two guides demonstrate how a Spring Batch Steps can be partitioned on Hadoop YARN. The second guide show how a failed Batch job can be restarted and how only the remaining steps are executed on Hadoop YARN.
* Batch YARN Application
* Restartable Batch YARN Application

comments powered by Disqus