Spring Team
Mark Pollack

Mark Pollack

Spring Cloud Data Flow co-lead

New York, NY

Mark Pollack is a software engineer with Pivotal and is the co-lead of the Spring Cloud Data Flow and Spring XD projects He has been a contributor to many Spring projects dating back to the Spring Framework in 2003 as well as founding the Spring.NET and Spring Data projects
Blog Posts by Mark Pollack

Spring Cloud Data Flow 1.5.0 Released

The Spring Cloud Data Flow team is pleased to announce the 1.5.0 GA release. Follow the Getting Started guides for Local Server, Cloud Foundry, and Kubernetes.

Here are the highlights:

  • UI Improvements

  • Spring Boot, Spring Cloud Stream 2.0, and Spring Cloud Task 2.0 Support

  • Updated Application Starters

  • Metrics Improvements

  • Nested splits for Composed Tasks

  • Kubernetes Improvements

  • Updated File Ingest sample

UI Improvements

We have continued to improve the UI/UX of the Dashboard. We hope that you will immediately notice an overall lighter weight design. The Tasks tab has been rewritten to match the UX styling of the other tabs. A new paginator component has been added to all the list pages. Switching from a list of 20, 30, 50, or 100 items per page is possible. This further simplifies the bulk operation workflows.

The updated Stream Builder tab makes is easy to deploy Stream Definitions and update deployed streams. You can edit application and deployment properties as well as change the version of individual applications in the stream and re-deploy. Data Flow’s integration with Skipper handles the upgrade process, allowing for easy rollback in case the upgrade doesn’t go as planned. The Stream Builder tab also includes many optimizations, including better form validation and eager error reporting. Try it out!

Stream Builder Tab

There has also been a significant amount of refactoring to optimize the code base and prepare for future extensions and feature additions. End-to-end testing with Selenium and SauceLabs has also been added.

Read more...

Spring Cloud Data Flow 1.5 RC1 released

The Spring Cloud Data Flow team is pleased to announce the release of 1.5.0 RC1. Follow the Getting Started guides for Local Server, Cloud Foundry, and Kubernetes.

Here are the highlights:

General Improvements

  • Switch to Hikari connection pool and restructure code to use fewer connections.

  • Several bug fixes in underling deployer libraries.

Dashboard

  • Editing a created/deployed stream is now possible from the Stream Builder. The application and deployment properties can be edited and re-deployed. The App version can be switched, too.

  • A new paginator component is added to all the list page. Switching from a list of 20, 30, 50, or 100 items per page is possible. This further simplifies the bulk operation workflows.

  • Introduction of end-to-end testing via Selenium and SauceLabs.

Read more...

Spring Cloud Data Flow 1.5 M1 released

The Spring Cloud Data Flow team is pleased to announce the release of 1.5.0 M1. Follow the Getting Started guides for Local Server, Cloud Foundry, and Kubernetes.

Here are the highlights:

  • UI Improvements

  • Spring Boot & Spring Cloud Stream 2.0 Support

  • Nested splits for Composed Tasks

  • Metrics Collector 2.0 M1

  • Stream Application Starters Darwin M1 release train

  • Support for deploying to multiple Kubernetes clusters

UI Improvements

We have continued to improve the UI/UX of the Dashboard. You will immediately notice an overall lighter weight design. The Tasks tab has been rewritten to match the UX styling of the other tabs. The stream-builder view includes many optimizations ranging from better form validation and eager error reporting. Try it out!

There has also been a significant amount of refactoring to optimize the codebase and prepare for future extensions and feature additions.

Read more...

Spring Cloud Skipper 1.0 GA Released

On behalf of the team, I am pleased to announce the release of Spring Cloud Skipper 1.0 GA

Skipper is a lightweight tool that allows you to discover Spring Boot applications and manage their lifecycle on multiple Cloud Platforms. You can use Skipper standalone or integrate it with Continuous Integration pipelines to help implement the practice of Continuous Deployment.

The getting started section in the reference guide is the best place to start kicking the tires.

Release Highlights:

  • Introduction of Flyway to manage schema along with various schema tweaks.

  • Option to delete a release along with its package.

  • Refined the REST API.

  • Updated properties to YAML converter.

  • Add resource metadata in manifest template.

  • Separate platform deployers into multiple maven modules.

  • Support passing to the shell commands to execute.

  • Updated documentation.

  • Various bug fixes.

Read more...

Spring Cloud Data Flow 1.3.0.RC1 released

We are pleased to announce the 1.3.0.RC1 release of the Spring Cloud Data Flow and its associated ecosystem of projects.

Follow the Getting Started guides for Local Server, Cloud Foundry, and Kubernetes.

Release Highlights

Feature toggle for Skipper

To simplify the overall experience of opting into using Skipper to deploy streams, a feature toggle provides you the ability to switch between skipper mode and the previous 'classic' mode. The feature toggle is used in both the Shell and the Server. The default value is to use the 'classic' non-skipper mode. To enable skipper mode, pass in the following Spring Boot options.

java -jar spring-cloud-dataflow-server-local-1.3.0.RC1.jar  \
  --spring.cloud.dataflow.features.skipper-enabled=true

java -jar spring-cloud-dataflow-shell-1.3.0.RC1.jar --dataflow.mode=skipper

This also allowed us to remove stream skipper commands introduced in 1.3 M3 and collapse skipper behavior into stream commands.

Read more...

Spring Cloud Skipper 1.0 RC1 Released

On behalf of the team, I am pleased to announce the release of Spring Cloud Skipper 1.0 RC1.

Skipper is a lightweight tool that allows you to discover Spring Boot applications and manage their lifecycle on multiple Cloud Platforms. You can use Skipper standalone or integrate it with Continuous Integration pipelines to help implement the practice of Continuous Deployment.

The 1.0 RC1 release fixes several bugs and introduces a some new features.

  • OAuth Security support.
  • Release install, upgrade, and rollback workflow managed using the Spring StateMachine project.
  • REST API improvements.
  • Database Schemas managed using Flyway.
  • Package deletion, checking for active Releases.
  • Release deletion with optional package deletion.
  • Shell commands follow a consistent format, e.g. platform list, release status.
  • Add support for ResourceMetadata URIs in package template.
  • Support for interactive and non-interactive shell modes.
  • Improved conversion from java.util.Properties, to YAML in shell.
Read more...

Spring Cloud Data Flow for Cloud Foundry 1.3.0.M3 released

We are pleased to announce the 1.3.0.M3 release of the Spring Cloud Data Flow for Cloud Foundry.

The Getting Started Guide is the best place to start kicking the tires.

Release Highlights

Stream updates, a JavaDSL, and the complete port of the UI to the Angular 4.0 stack are some of the main highlights. More information on release highlights can be found in the release blog for the core Data Flow project.

Of note for the Cloud Foundry server is an upgrade to v2.23.0 of the cf-java client library and setting the default health check to be http instead of port. You can now also specify the health check endpoint URL and timeout values as deployment properties.

Read more...

Spring Cloud Data Flow 1.3.0.M3 released

We are pleased to announce the 1.3.0.M3 release of the Spring Cloud Data Flow and its associated ecosystem of projects.

Local Server: Getting Started Guide

Release Highlights

Stream updates and rollback

A streaming data pipeline orchestrated as a series of microservice applications has always been the core value of Spring Cloud Data Flow’s design. In 1.3.0.M3 we have provided the ability to update sources, processors, and sinks independently without having to undeploy and redeploy the entire stream.

The stream update feature is implemented by delegating the deployment process to a new Spring Cloud project called Skipper. Introduced in this blog, Spring Cloud Skipper is a standalone server that deploys Spring Boot applications to multiple cloud platforms. It also keeps track of the application version, application properties, and deployment properties of the deployed application or applications so that the changes to any of these properties can be calculated upon an update request.

When a request is made to update applications in a Stream, only the application or applications that need to be changed are updated. A simple red/black update is performed and the design relies on Spring Boot’s health endpoint associated with the new application(s). By keeping track of the deployments, Skipper can also rollback to previous versions of the stream.

The following new Data Flow commands show the basic lifecycle of updating the log sink from version 1.1.0 to version 1.2.0 and then rolling back to 1.1.0. This assumes that both version 1.1.0 and 1.2.0 of the log sink were deployed as maven artifacts. Note that the HTTP source application remains deployed throughout the changes to the log sink.

dataflow:> app register --name http --type source --uri maven://org.springframework.cloud.stream.app:http-source-rabbit:1.2.0.RELEASE

dataflow:> app register --name log --type sink --uri maven://org.springframework.cloud.stream.app:log-sink-rabbit:1.1.0.RELEASE

dataflow:> stream create --name httptest --definition "http --server.port=9000 | log"

dataflow:> stream skipper deploy --name httptest

dataflow:> stream skipper update --name httptest --properties version.log=1.2.0.RELEASE

dataflow:> stream skipper rollback --name httptest

The Getting Started and Streams with Skipper sections of the documentation walk through the process in much greater detail.

Read more...

Spring Cloud Skipper 1.0 M2 Released

On behalf of the team, I am pleased to announce the release of Spring Cloud Skipper 1.0 M2.

Skipper is a lightweight tool that allows you to discover Spring Boot applications and manage their lifecycle on multiple Cloud Platforms. You can use Skipper standalone or integrate it with Continuous Integration pipelines to help implement the practice of Continuous Deployment.

The 1.0 M2 release fixes several bugs and introduces a few new features.

  • Support for Postgres, MySQL, Microsoft SQL Server, and HSQLDB databases.
  • Improved support for upgrading applications that use an HTTP location for the resource definition.
  • LRU cache used to manage disk space for HTTP and Maven based resources that are downloaded.
  • HTTP based resources are always downloaded, never cached.
  • Use updated CF Deployer library with an HTTP based health check.
Read more...

Spring Cloud Skipper 1.0 M1 Released

On behalf of the team, I am pleased to announce the release of Spring Cloud Skipper 1.0 M1.

Skipper is a lightweight tool that allows you to discover Spring Boot applications and manage their lifecycle on multiple Cloud Platforms. You can use Skipper standalone or integrate it with Continuous Integration pipelines to help implement the practice of Continuous Deployment.

The main features in Skipper 1.0 M1 are:

  • Define multiple platform accounts where Spring Boot applications can be deployed. Supported platforms are Local, Cloud Foundry, and Kubernetes.
  • Substitute variables in Mustache templated files that describe how to deploy applications to a platform.
  • Search Package Repositories for existing applications.
  • Upgrade/Rollback a package based on a simple blue/green workflow.
  • Store the history of resolved template files (aka ‘application manifests’) which represent the final description of what has been deployed to a platform for a specific release.
  • Use via a standalone interactive shell or web API.
Read more...