This post is part of a blog series that explores the newly redesigned Spring Cloud Stream applications based on Java Functions. This episode, presented in two parts, explores strategies for testing functions used to implement stream applications. We will pay special attention to functions that integrate with external resources, which presents additional testing challenges. Such is the case with most of the pre-packaged source and sink applications. To illustrate this, we will walk through a sample couchbase-sink application. Here in Part 1, we will focus on the core function on which the sink…

This is Part 2 of Testing Stream Applications. In Part 1 we implemented and tested the core function needed for our sample couchbase-sink application. The tests at the function level covered expected success and error scenarios and relied on Testcontainers to provision a Couchbase cluster. This post assumes you have read Part 1 and continues where it left off. Couchbase Sink In Part 1 we verified that the function we wrote for upserting data into Couchbase works as expected. We can now use the function, exposed as a java.util.Consumer, to implement a sink to use in a data pipeline built with…

This article is part of a blog series that explores the newly redesigned Spring Cloud Stream applications based on Java Functions. In this chapter, we explore how to use Spring Cloud Stream Applications and Spring Cloud Data Flow to implement a very common ETL use case: Ingesting files from a remote service. Specifically, we will look at how to ingest files from S3, SFTP, and FTP. Here is what is included in the blog series to date: Introducing Function Based Streaming Applications Function Composition with Streaming Applications How to Build a Supplier and Source Application How to Build a…

Introduction We began this series by introducing the new stream applications based on Java functions, and function composition. The previous entry presented a tutorial for building a simple stream application and running it in Spring Cloud Data Flow. Today we explore the HTTP Request Function and present examples of how to use it. In case you missed it, the prior posts in this series are: Introducing Function Based Streaming Applications Function Composition with Streaming Applications How to Build a Supplier and Source Application How to Build a Consumer and Sink Application Build and Run a…

Introduction So far in this series we have introduced the new stream applications based on Java functions, and function composition. We have also provided detailed examples of how to build a source from a supplier and a sink from a consumer. Here we continue the journey with the first of several case studies to follow. Each case study demonstrates how to use one or more of the available pre-packaged Spring Boot stream applications, in various scenarios, to build data streaming pipelines. Today we will showcase two of the most commonly used applications, the HTTP source and the JDBC sink. We…

Introducing Java Functions for Spring Cloud Stream Applications - Part 1 Last week we posted Introducing Java Functions for Spring Cloud Stream Applications - Part 0 to announce the release of Spring Cloud Stream applications 2020.0.0-M2. Here, we explore function composition, one of the more powerful features enabled by the function oriented architecture presented in Part 0. If you haven’t had a chance to read Part 0, now would be a great time! Function Composition Function composition has a solid theoretical foundation in mathematics and computer science. In practical terms, it is a way to…

We are happy to announce the release of Spring Cloud Stream applications 2020.0.0-M2. This release is a complete overhaul of the legacy Spring Cloud Stream App Starters. Starting with this release, we are moving away from theme-oriented release train names (famous scientists in alphabetical order) to calendar based versioning. The current GA release is called Einstein, and we are pleased to introduce 2020.0.0-M2. We are also moving away from the app starters. Having reorganized, repackaged, and (in some cases) rewritten the underlying code, we now have a new Git repository: spring-cloud/stream…

This post is intended to introduce the Groovy Bean Builder to Java developers as a powerful alternative or supplement to Java @Configuration and XML configuration. The Spring Framework release 4.0 includes a port of the Grails Bean Builder to the core Spring Framework, providing a Groovy DSL for configuring Spring applications. Groovy and Grails developers are no doubt familiar with configuring Spring applications this way and I expect the rest of you are already thinking "How cool is that?" Don't worry if you're not a Groovy expert. Just as many Java programmers use another popular Groovy…

I am pleased to announce the GA release of Spring Data GemFire 1.3.0. In addition to many minor bug fixes and enhancements, this release includes some notable new features to make writing Java applications with GemFire even easier: The above configuration will create a client cache, pool, and proxy client regions for all available regions on the server, with sensible defaults, and register them as Spring beans.

Spring Integration implements Enterprise Integration Patterrns using the Spring programming model to enable messaging in Spring-based applications. Spring Integration also provides integration with external systems using declarative adapters supporting jms, http, amqp, tcp, ftp(s), smtp, and so on. Currently, configuring message flows is primarily done via Spring XML and Spring Integration supports several namespaces to make this as succinct as possible. Earlier this year, SpringSource released a Scala DSL for Spring Integration. Now, we are pleased to announce the first milestone release…