close

Ilayaperumal Gopinathan

Ilayaperumal Gopinathan

Spring Cloud Data Flow Co-Lead

Madurai, India

Ilayaperumal (a.k.a. Ilaya) Gopinathan is a co-lead of Spring Cloud Data Flow project. Prior to joining the Spring team, he worked on products such as vFabric App Suite, Cloud Foundry, and Hyperic at VMware.
Blog Posts by Ilayaperumal Gopinathan

Spring Cloud Data Flow 2.7.0-M1 and 2.6.3 Released

Spring Cloud Data Flow team is pleased to announce the release of 2.7.0-M1 and 2.6.3

While the release 2.6.3 has the bug fixes, the first milestone of 2.7.0 has the following enhancements:

SCDF dashboard re-design using VMware Clarity toolkit

SCDF dashboard is re-designed using VMware Clarity design system. This is a complete rewrite of SCDF dashboard GUI to get the benefits of the Clarity design.

Spring Boot 2.3.2 Update

SCDF server components are now updated to use Spring Boot 2.3.2 release.

Wavefront integration

Read more...

Spring Cloud Data Flow 2.6.0-M1 Released

Spring Cloud Data Flow team is pleased to announce the first milestone release of 2.6.0.

This 2.6.0-M1 adds some bug fixes and the following features:

  • Composed Task Runner as SCDF native module
    Composed Task Runner is now an integral part of SCDF itself and gets registered implicitly while the users who have the custom composed task runner can still override the native one. This enables us to integrate the composed tasks management within the context of Spring Cloud Data Flow. This change also allowed us to provide some additional user experiences when managing composed tasks. .

  • Improvements in task applications management
    This milestone added some improvements at the backend such as the ability to uniquely identify the task elements inside the composed task, adding task manifest as part of the last ran task definition, ability to cleanup task related resources as part of task definition cleanup along with the SCDF dashboard changes.

  • Liveness/Readiness probes for SCDF/Skipper deployments
    SCDF/Skipper server Kubernetes deployment configuration files are updated with the liveness/readiness probes which can be customized based on the deployment setup.

  • Bundling SCDF/Skipper DB schema files
    The DDL scripts for all the supported databases are now bundled and available in the classpath. For the database administrators who want to setup the database without the flyway activated schema management, this bundle would come in handy.

Read more...

Spring Cloud Data Flow 2.5.0.RC1 Released

Spring Cloud Data Flow team is pleased to announce the first milestone release of 2.5.0.RC1.

This first release candidate of 2.5.0 adds some bug fixes and the following features:

  • Support application metadata as part of the Container images. Before this release, we only supported application property whitelisting when using a maven artifact.

  • Support for Azure AD integration using OAuth 2.0. Spring Cloud Data Flow, Spring Cloud Skipper servers along with the Spring Cloud Data Flow shell can be configured to integrate with Azure AD for authentication and authorization.

  • Removal of Scheduler task launcher component when scheduling tasks. In Spring Cloud Data Flow 2.3.x, the scheduling of tasks was designed to have an intermediate scheduler task launcher component to enhance the continuous deployment user experience. Since this component added some additional complexities and overhead on Kubernetes, the 2.5.x release removed this intermediary task launcher. We also added a migration tool to migrate the schedules from 2.3.x and 2.4.x

  • Unification of Kubernetes Scheduler properties. This feature lets the user configure any supported/applicable task deployment properties at the time of scheduling them as well.

Read more...

Spring Cloud Data Flow 2.5.0.M1 Released

Spring Cloud Data Flow team is pleased to announce the first milestone release of 2.5.0.M1.

The first milestone release of 2.5.0 consists of performance improvements addressed at application status retrieval for streams in Cloud Foundry. This also involves revamping the runtime application view page along with better pagination for streams at the SCDF dashboard. You can see more information on these improvements on the recently released SCDF 2.4.2 GA.

This milestone release also consists of some important bug fixes mentioned in here

Read more...

Spring Cloud Data Flow 2.4.2 GA Released

Spring Cloud Data Flow team is pleased to announce the GA release of 2.4.2.

In this release, along with some bug fixes, we primarily focused on the server/client-side performance improvements on the stream management.

The REST layer in Spring Cloud Data Flow server is refactored to accommodate better pagination and performance when retrieving the runtime application statuses of streams. In addition to that, the SCDF dashboard pages are re-worked based on these changes. The Runtime Applications page in the Dashboard has a new layout where the runtime applications are grouped by their streams. The Streams page is also refactored to efficiently retrieve the stream status, application metrics, etc.,

Read more...

Spring Cloud Data Flow 2.3.0.M1 and 2.2.1 GA Released

The Spring Cloud Data Flow team is pleased to announce the milestone release 2.3.0.M1 and the release of 2.2.1 GA.

Along with the bug fixes and enhancements, the first milestone of 2.3.0 also has the following new features:

  • Improved Prometheus monitoring based on Prometheus RSocket. It uses persistent bidirectional (RSocket) connections between the Stream/Task applications and a Prometheus RSocket Proxy instance. When Prometheus scrapes the proxy instance, and the proxy, in turn, uses the connections to pull metrics from the applications.
  • Ability to pass OAuth Access Token as command-line arguments for Composed Tasks
  • Users can now specify a description of the stream when creating it. Thanks to Daniel Serleg for the contribution!
  • Spring Cloud Scheduler SPI is now merged into Spring Cloud Deployer. We’ll deprecate the explicit spring-cloud-scheduler project and its implementations on Cloud Foundry, K8s and have them available as part of spring-cloud-deployer.
Read more...

Spring Cloud Data Flow 2.2.0.M1 Released

The Spring Cloud Data Flow team is pleased to announce the milestone release 2.2.0.M1.

Along with some enhancements and bug fixes, here are some of the highlights of the release:

Metrics for Tasks
Metrics and monitoring continue to be one of the important themes for Spring Cloud Data Flow. Building upon the InfluxDB integration that we have for streams, in 2.2, we are adding support for Tasks, too. The core of the Micrometer integration landed in Spring Cloud Task’s 2.2.0 release-line, which by the way is a requirement if you are going to try out the Task-metrics and the SCDF integration.

Read more...

Spring Cloud Data Flow 2.1.2 GA and 2.0.3 GA Released

The Spring Cloud Data Flow team is pleased to announce the release of 2.1.2 GA and 2.0.3 GA of Data Flow.

This is a minor release with mostly bug fixes and documentation enhancements.

Here are the highlights of the releases:

  • Bug fixes on Spring Cloud Data Flow, Spring Cloud Data Flow UI and Spring Cloud Scheduler for CloudFoundry

  • Java Stream DSL now supports stream update operation

  • Spring Cloud Deployer now uses Maven Resolver instead of Aether for resolving the Maven artifacts

Along with the releases, we also have the Spring Cloud Data Flow website enhanced with

Read more...

Spring Cloud Data Flow 2.0.2 GA Released

The Spring Cloud Data Flow team is pleased to announce the release of 2.0.2 GA of Data Flow. Follow the Getting Started guides for running on Local, Cloud Foundry, and Kubernetes.

This is a minor release with mostly bug fixes and documentation enhancements.

Here are the highlights of this release:

  • Improved documentation

    • Docs update for enabling Kafka on Helm Chart

    • Improved architecture diagrams

    • Getting started guide improvements

  • Bug fixes on Spring Cloud Data Flow, Spring Cloud Data Flow UI, and Spring Cloud Deployer implementations

Read more...

Spring Cloud Data Flow 2.0.1 GA Released

The Spring Cloud Data Flow team is pleased to announce the release of 2.0.1 GA of Data Flow. Follow the Getting Started guides for running on Local, Cloud Foundry, and Kubernetes.

This is a minor release with mostly bug fixes and documentation enhancements.

Here are the highlights of this release:

  • Kubernetes deployment configuration for Skipper server is now backed by MySQL as the default database.

  • Remove extraneous Spring Data Redis dependency to fix Spring Cloud Data Flow server health check.

  • Improved Getting Started documentation.

  • Bug fixes on Spring Cloud Data Flow and Spring Cloud Data Flow UI.

  • All the out-of-the-box stream apps from the Documentation examples now use Einstein SR2.

Read more...