Recorded at SpringOne2GX 2015. Track: Big Data Speaker: Marius Bogoevici Slides: http://www.slideshare.net/SpringCentral/stream-processing-at-scale-with-spring-xd-and-kafka In the recent years, drastic increases in data volume, as well as a greater demand for low latency have led to a radical shift in business requirements and application development methods. Near-realtime data processing has started to become more prevalent, and high-throughput messaging systems such as Apache Kafka have emerged as key building blocks. Focusing on developer experience and productivity, Spring XD makes it easy to develop big data applications, without the need for dealing with the details of integrating and scaling a big data stack. In the particular context of Kafka, this means allowing developers to benefit from its specific features and power, while at the same time remaining focused on writing and testing business logic. To begin, we will provide a brief introduction of how Kafka is supported in the Spring ecosystem in general, in Spring Integration and Spring Data, and then we will discuss how Spring XD integrates with Kafka as an external datasource and transport. And because we like all things practical, the core part of the presentation will walk you through a demo that will show you how to unleash the power of Kafka with Spring XD, by building a highly scalable data pipeline with RxJava and Kafka, using Spring XD as a platform.