Spring Team
Mark Pollack

Mark Pollack

Spring XD co-lead/Spring Data Lead

New York, NY

Mark Pollack is a software engineer with Pivotal and is the co-lead of the Spring Cloud Data Flow and Spring XD projects He has been a contributor to many Spring projects dating back to the Spring Framework in 2003 as well as founding the Spring.NET and Spring Data projects.
Blog Posts by Mark Pollack

Spring Cloud Data Flow 1.1 GA released

On behalf of the team, I am pleased to announce the GA release of Spring Cloud Data Flow 1.1. Follow the links in the getting started guide to download the local server implementation and shell to create Stream and Tasks.

General highlights of the 1.1 GA Release include:

Read more...

Spring Cloud Data Flow 1.1 RC1 Released

On behalf of the team, I am pleased to announce the first release candidate of Spring Cloud Data Flow 1.1. Follow the links in the getting started guide to download the local server implementation and shell to create Stream and Tasks.

The 1.1 RC1 release includes the following new features and improvements:

  • Builds upon Camden.SR2 release improvements

  • LDAP authentication is now supported with SSL

  • Portable deployment properties for memory and cpu are in place for support across various runtime implementations

  • Passing Java Options to the local JVM when launching application is now supported

  • UI Improvements

    • List pages now support sorting

    • Server-side search support for stream and task list pages

    • Content-assist for bulk task definitions including the support for incremental validations of task application properties

  • Add content assist support for tasks in the shell

  • Thanks to the community for adding DB2 support for the TaskRepository

  • Documentation on how to use Spring Boot Admin to visualize server metrics

Read more...

Spring Cloud Data Flow 1.1 M2 Released

On behalf of the team, I am pleased to announce the release of the second milestone of Spring Cloud Data Flow 1.1. You can download the local server that is part of this release here.

The 1.1 M2 release includes the following new features and improvements:

  • Builds upon Boot 1.4.1 and Spring Cloud Camden improvements

  • Task application properties can now be referenced using non-prefixed property names

  • Add visual representation for related streams. This representation also includes nested TAPs and the downstream processing nodes in an overall topology view.

Read more...

Spring Cloud Data Flow 1.0 GA released

On behalf of the team, I’m excited to announce the 1.0 GA release of Spring Cloud Data Flow!

Note
A great way to start using this new release is to follow the Getting Started section of the reference documentation. It uses a Data Flow server that runs on your computer and deploys a new process for each application.

Spring Cloud Data Flow (SCDF) is an orchestration service for data microservices on modern runtimes. SCDF lets you describe data pipelines that can either be composed of long lived streaming applications or short lived task applications and then deploys these to platform runtimes that you may already be using today, such as Cloud Foundry, Apache YARN, Apache Mesos, and Kubernetes. We provide a wide range of stream and task applications so you can get started right away to develop solutions for use-cases such as data ingestion, real-time analytics and data import/export.

Read more...

Spring Cloud Data Flow 1.0 RC1 released

On behalf of the team I am pleased to announce the release of Spring Cloud Data Flow 1.0 RC1.

Several exciting new features have been added in this release which carry over to the other Data Flow Server implementations that were also released today.

Follow the links above for details on features unique to each individual runtime platform. The highlights of the release are:

Read more...

Spring XD 1.3.1 GA and Flo for Spring XD 1.0.1 GA released

Today we are pleased to announce the general availability of Spring XD 1.3.1 and Flo for Spring XD 1.0.1

  • Spring XD 1.3.1 GA: zip, brew and rpm.
  • Flo for Spring XD 1.0.1 GA: zip.

Here are some highlights of bug fixes and general improvements. Consult the JIRA release notes for the full list of issues fixed.

Read more...

Spring Cloud Stream 1.0 M3 and Data Flow 1.0 M2 released

On behalf of the team, I am pleased to announce several releases in the Spring Cloud Stream and Spring Cloud Data Flow family of projects.

Spring Cloud Stream 1.0 M3 introduces the following features

Spring Cloud Stream Modules 1.0 M2 adds many new modules with updated documentation.

  • Sources: File, Load Generator, sftp, and tcp
  • Processors: httpclient, PMML, and Splitter
  • Sinks: Cassandra, Field Value Counter, file, ftp, gemfire, HDFS Dataset, JDBC, tcp, throughput, and websocket
Read more...

Spring XD 1.3 GA and Flo for Spring XD 1.0 GA released

Today we are pleased to announce the general availability of Spring XD 1.3 and Flo for Spring XD 1.0.

  • Spring XD 1.3 GA: zip, brew and rpm.
  • Flo for Spring XD 1.0 GA: zip.

In addition to bug fixes we have also added several new features in the 1.3 release line

  • Job Composition DSL allows for the creation of a complex graph of job executions.
  • Flo for Spring XD designer supports creating composed jobs.
  • Admin UI supports execution history of composed jobs.
  • Cassandra Sink and Header Enricher Processor
  • Gpfdist sink now supports update operations and full range of control file options
  • Spark 1.3.1 Support
  • A timeout value for flushing writes to HDFS in order to ensure data is persisted on the HDFS DataNode’s disks.
  • General dependency upgrades, Spring Data Gosling, SI 4.2, and Boot 1.2
  • Hadoop distribution version updates to Apache Hadoop 2.7.1 and Hortonworks Data Platform 2.2. Pivotal Hadoop 2.1, 3.0 and Cloudera Hadoop 5.3
Read more...

Spring XD 1.3 RC1 released

On behalf of the Spring XD team, I am very pleased to announce the first release candidate of Spring XD 1.3 is now available for download. You can also install using brew and rpm.

This release includes some major new functionality for batch jobs. We have introduced the ability to create composed jobs that allows you to create a complex graph of jobs executed based on a new Job DSL. Flo for Spring XD UI has been updated to support the new Job DSL and provide a visual drag and drop canvas for creating composed jobs. Spring XD’s job execution UI also supports execution history of composed jobs. The video in the Flo for XD blog post shows it in action.

Read more...

Spring XD 1.3 M1 released

On behalf of the Spring XD team, I am very pleased to announce the first milestone release of Spring XD 1.3

This release includes bug fixes and some enhancements:

  • Hadoop distribution version updates to Apache Hadoop 2.7.1 and Hortonworks Data Platform 2.3. Pivotal Hadoop 2.1, 3.0 and Cloudera Hadoop 5.4
  • Spark 1.3.1 Support
  • Cassandra Sink
  • Header Enricher Processor
  • Bug fixes to the Flo Designer UI and Admin UI
  • Gpfdist sink now supports update operations and full range of control file options
  • Update to Spring Integration 4.2 which provides performance improvements when monitoring is enabled.
  • Upgrade to Spring Data Gosling Release train
Read more...