Spring Batch 4.1.0.M1 Released
We are pleased to announce that Spring Batch 4.1.0.M1 is now available on Github and the Pivotal download repository. Many thanks to all of those who contributed to this release!
What’s new?
Here are the highlights of this release:
- Simplify testing
- Simplify remote chunking
- Add a new JSON item reader
Simplify Testing
Spring Batch provides some nice utility classes (such as the JobLauncherTestUtils
and JobRepositoryTestUtils
) and test execution listeners (StepScopeTestExecutionListener
and JobScopeTestExecutionListener
) to test batch components. However, in order to use these utilities, you must configure them explicitly, as shown in the following example:
@RunWith(SpringRunner.class)
@ContextConfiguration(classes = {JobConfiguration.class, JobTest.TestConfiguration.class})
@TestExecutionListeners(listeners = {StepScopeTestExecutionListener.class, JobScopeTestExecutionListener.class})
public class JobTest {
@Autowired
private JobLauncherTestUtils jobLauncherTestUtils;
@Autowired
private JobRepositoryTestUtils jobRepositoryTestUtils;
@Before
public void clearMetadata() {
jobRepositoryTestUtils.removeJobExecutions();
}
@Test
public void testJob() throws Exception {
// given
JobParameters jobParameters =
jobLauncherTestUtils.getUniqueJobParameters();
// when
JobExecution jobExecution =
jobLauncherTestUtils.launchJob(jobParameters);
// then
Assert.assertEquals(ExitStatus.COMPLETED,
jobExecution.getExitStatus());
}
@Configuration
public static class TestConfiguration {
@Bean
public JobLauncherTestUtils jobLauncherTestUtils() {
return new JobLauncherTestUtils();
}
@Bean
public JobRepositoryTestUtils jobRepoTestUtils(DataSource dataSource,
JobRepository jobRepository) {
return new JobRepositoryTestUtils(jobRepository, dataSource);
}
}
}
In this release, we introduced a new annotation, named @SpringBatchTest
, that marks a class as a test class for Spring Batch components. This annotation automatically adds utility beans and listeners to the test context and makes them automatically available for autowiring, as the following example shows:
@RunWith(SpringRunner.class)
@SpringBatchTest
@ContextConfiguration(classes = {JobConfiguration.class})
public class JobTest {
@Autowired
private JobLauncherTestUtils jobLauncherTestUtils;
@Autowired
private JobRepositoryTestUtils jobRepositoryTestUtils;
@Before
public void clearMetadata() {
jobRepositoryTestUtils.removeJobExecutions();
}
@Test
public void testJob() throws Exception {
// given
JobParameters jobParameters =
jobLauncherTestUtils.getUniqueJobParameters();
// when
JobExecution jobExecution =
jobLauncherTestUtils.launchJob(jobParameters);
// then
Assert.assertEquals(ExitStatus.COMPLETED,
jobExecution.getExitStatus());
}
}
With this new annotation, you can declaratively import test utilities and focus on testing the business logic of the batch job.
Simplify Remote Chunking
Setting up a remote chunking job requires the definition of a number of beans:
- A connection factory to acquire connections from the messaging middleware (JMS, AMQP, and others)
- A
MessagingTemplate
to send requests from the master to the workers and back again - An input channel and an output channel for Spring Integration to get messages from the messaging middleware
- A special item writer (
ChunkMessageChannelItemWriter
) on the master side that knows how to send chunks of data to workers for processing and writing - A message listener (
ChunkProcessorChunkHandler
) on the worker side to receive data from the master
This can be a bit daunting at first glance. In this milestone, we tried to make this task as easy as possible by introducing a new annotation named @EnableBatchIntegration
as well as new APIs (RemoteChunkingMasterStepBuilder
and RemoteChunkingWorkerBuilder
) to simplify the configuration. Here is an example:
@Configuration
@EnableBatchProcessing
@EnableBatchIntegration
public class RemoteChunkingAppConfig {
@Autowired
private RemoteChunkingMasterStepBuilderFactory masterStepBuilderFactory;
@Autowired
private RemoteChunkingWorkerBuilder workerBuilder;
@Bean
public TaskletStep masterStep() {
return this.masterStepBuilderFactory
.get("masterStep")
.chunk(100)
.reader(itemReader())
.outputChannel(outgoingRequestsToWorkers())
.inputChannel(incomingRepliesFromWorkers())
.build();
}
@Bean
public IntegrationFlow worker() {
return this.workerBuilder
.itemProcessor(itemProcessor())
.itemWriter(itemWriter())
.inputChannel(incomingRequestsFromMaster())
.outputChannel(outgoingRepliesToMaster())
.build();
}
}
This new annotation and builders takes care of the heavy lifting of configuring infrastructure beans. You can now easily configure a master step as well as a Spring Integration flow on the worker side.
Add a New JSON Item Reader
JSON is a popular data format nowadays, and many applications have a requirement to read and write JSON data in batch mode. In this first milestone, we have added a new item reader that supports JSON. Similar to the StaxEventItemReader
for XML, the JsonItemReader
uses streaming APIs in order to read JSON objects in chunks. Two JSON libraries are supported: Jackson and Gson. The JsonItemReader
is able to read JSON files in the following format:
[
{
"isin": "123",
"quantity": 1,
"price": 1.2,
"customer": "foo"
},
{
"isin": "456",
"quantity": 2,
"price": 1.4,
"customer": "bar"
}
]
Each object of this trades.json
file represents an instance of the following Trade
class:
public class Trade {
private String isin;
private long quantity;
private BigDecimal price;
private String customer;
// getters and setters omitted
}
In order to read this trades.json
file, you can use the following item reader (using Jackson in this example):
@Bean
public JsonItemReader<Trade> jsonItemReader() {
ObjectMapper objectMapper = new ObjectMapper();
// configure the objectMapper as required
JacksonJsonObjectReader<Trade> jsonObjectReader =
new JacksonJsonObjectReader<>(Trade.class);
jsonObjectReader.setMapper(objectMapper);
return new JsonItemReaderBuilder<Trade>()
.jsonObjectReader(jsonObjectReader)
.resource(new ClassPathResource("trades.json"))
.name("tradeJsonItemReader")
.build();
}
Other Improvements
This release also includes many other improvements, bug fixes, and documentation updates. For a complete list of changes, please check the change log. We look forward to hearing your feedback on this milestone! Please feel free to ping @michaelminella, @benas, or @cppwfs on twitter or ask your question on StackOverflow or Gitter. If you find any issue, please open a ticket on Jira.
What’s Next?
In the next milestone, we plan to:
- Simplify remote partitioning setup (similar to simplifying remote chunking setup in this milestone)
- Add a
JsonItemWriter
to complement theJsonItemReader
- Add JSR-305 annotations to public APIs
Stay tuned!
Spring Batch Home | Source on GitHub | Reference Documentation