We are pleased to announce that Spring Batch 4.1.0.M1 is now available on Github and the Pivotal download repository. Many thanks to all of those who contributed to this release!

What's new?

Here are the highlights of this release:

• Simplify testing
• Simplify remote chunking
• Add a new JSON item reader

Simplify Testing

Spring Batch provides some nice utility classes (such as the JobLauncherTestUtils and JobRepositoryTestUtils) and test execution listeners (StepScopeTestExecutionListener and JobScopeTestExecutionListener) to test batch components. However, in order to use these utilities, you must configure them explicitly, as shown in the following example:

@RunWith(SpringRunner.class)
@ContextConfiguration(classes = {JobConfiguration.class, JobTest.TestConfiguration.class})
@TestExecutionListeners(listeners = {StepScopeTestExecutionListener.class, JobScopeTestExecutionListener.class})
public class JobTest {

   @Autowired
   private JobLauncherTestUtils jobLauncherTestUtils;

   @Autowired
   private JobRepositoryTestUtils jobRepositoryTestUtils;


   @Before
   public void clearMetadata() {
      jobRepositoryTestUtils.removeJobExecutions();
   }

   @Test
   public void testJob() throws Exception {
      // given
      JobParameters jobParameters = 
            jobLauncherTestUtils.getUniqueJobParameters();

      // when
      JobExecution jobExecution = 
            jobLauncherTestUtils.launchJob(jobParameters);

      // then
      Assert.assertEquals(ExitStatus.COMPLETED, 
                          jobExecution.getExitStatus());
   }

   @Configuration
   public static class TestConfiguration {

      @Bean
      public JobLauncherTestUtils jobLauncherTestUtils() {
         return new JobLauncherTestUtils();
      }

      @Bean
      public JobRepositoryTestUtils jobRepoTestUtils(DataSource dataSource,
                                               JobRepository jobRepository) {

         return new JobRepositoryTestUtils(jobRepository, dataSource);
      }

   }
}

In this release, we introduced a new annotation, named @SpringBatchTest, that marks a class as a test class for Spring Batch components. This annotation automatically adds utility beans and listeners to the test context and makes them automatically available for autowiring, as the following example shows:

@RunWith(SpringRunner.class)
@SpringBatchTest
@ContextConfiguration(classes = {JobConfiguration.class})
public class JobTest {

   @Autowired
   private JobLauncherTestUtils jobLauncherTestUtils;

   @Autowired
   private JobRepositoryTestUtils jobRepositoryTestUtils;


   @Before
   public void clearMetadata() {
      jobRepositoryTestUtils.removeJobExecutions();
   }

   @Test
   public void testJob() throws Exception {
      // given
      JobParameters jobParameters = 
            jobLauncherTestUtils.getUniqueJobParameters();

      // when
      JobExecution jobExecution = 
            jobLauncherTestUtils.launchJob(jobParameters);

      // then
      Assert.assertEquals(ExitStatus.COMPLETED, 
                          jobExecution.getExitStatus());
   }
   
}

With this new annotation, you can declaratively import test utilities and focus on testing the business logic of the batch job.

Simplify Remote Chunking

Setting up a remote chunking job requires the definition of a number of beans:

• A connection factory to acquire connections from the messaging middleware (JMS, AMQP, and others)
• A MessagingTemplate to send requests from the master to the workers and back again
• An input channel and an output channel for Spring Integration to get messages from the messaging middleware
• A special item writer (ChunkMessageChannelItemWriter) on the master side that knows how to send chunks of data to workers for processing and writing
• A message listener (ChunkProcessorChunkHandler) on the worker side to receive data from the master

This can be a bit daunting at first glance. In this milestone, we tried to make this task as easy as possible by introducing a new annotation named @EnableBatchIntegration as well as new APIs (RemoteChunkingMasterStepBuilder and RemoteChunkingWorkerBuilder) to simplify the configuration. Here is an example:

@Configuration
@EnableBatchProcessing
@EnableBatchIntegration
public class RemoteChunkingAppConfig {
  
   @Autowired
   private RemoteChunkingMasterStepBuilderFactory masterStepBuilderFactory;
  
   @Autowired
   private RemoteChunkingWorkerBuilder workerBuilder;
  
   @Bean
   public TaskletStep masterStep() {
         return this.masterStepBuilderFactory
         	        .get("masterStep")
         	        .chunk(100)
         	        .reader(itemReader())
         	        .outputChannel(outgoingRequestsToWorkers())
         	        .inputChannel(incomingRepliesFromWorkers())
         	        .build();
   }
  
   @Bean
   public IntegrationFlow worker() {
         return this.workerBuilder
         	        .itemProcessor(itemProcessor())
         	        .itemWriter(itemWriter())
         	        .inputChannel(incomingRequestsFromMaster())
         	        .outputChannel(outgoingRepliesToMaster())
         	        .build();
   }
}

This new annotation and builders takes care of the heavy lifting of configuring infrastructure beans. You can now easily configure a master step as well as a Spring Integration flow on the worker side.

Add a New JSON Item Reader

JSON is a popular data format nowadays, and many applications have a requirement to read and write JSON data in batch mode. In this first milestone, we have added a new item reader that supports JSON. Similar to the StaxEventItemReader for XML, the JsonItemReader uses streaming APIs in order to read JSON objects in chunks. Two JSON libraries are supported: Jackson and Gson. The JsonItemReader is able to read JSON files in the following format:

[
  {
    "isin": "123",
    "quantity": 1,
    "price": 1.2,
    "customer": "foo"
  },
  {
    "isin": "456",
    "quantity": 2,
    "price": 1.4,
    "customer": "bar"
  }
]

Each object of this trades.json file represents an instance of the following Trade class:

public class Trade {

   private String isin;
   private long quantity;
   private BigDecimal price;
   private String customer;

   // getters and setters omitted

}

In order to read this trades.json file, you can use the following item reader (using Jackson in this example):

@Bean
public JsonItemReader<Trade> jsonItemReader() {

   ObjectMapper objectMapper = new ObjectMapper();
   // configure the objectMapper as required
   JacksonJsonObjectReader<Trade> jsonObjectReader = 
            new JacksonJsonObjectReader<>(Trade.class);
   jsonObjectReader.setMapper(objectMapper);

   return new JsonItemReaderBuilder<Trade>()
                 .jsonObjectReader(jsonObjectReader)
                 .resource(new ClassPathResource("trades.json"))
                 .name("tradeJsonItemReader")
                 .build();
}

Other Improvements

This release also includes many other improvements, bug fixes, and documentation updates. For a complete list of changes, please check the change log. We look forward to hearing your feedback on this milestone! Please feel free to ping @michaelminella, @benas, or @cppwfs on twitter or ask your question on StackOverflow or Gitter. If you find any issue, please open a ticket on Jira.

What’s Next?

In the next milestone, we plan to:

• Simplify remote partitioning setup (similar to simplifying remote chunking setup in this milestone)
• Add a JsonItemWriter to complement the JsonItemReader
• Add JSR-305 annotations to public APIs

Stay tuned!

Spring Batch Home | Source on GitHub | Reference Documentation