Supercharging Operations and Analytics: Using Spring XD to Support Analytics and CEP

News | Pieter Humphrey | March 15, 2016 | ...

Recorded at SpringOne2GX 2015. Speaker: Joseph Paulchell, CapitalOne Big Data Track Slides: http://www.slideshare.net/SpringCentral/supercharging-operations-analysis-using-spring-xd-to-support-analytics-and-cep Operations, analytics, and business teams are requesting ever increasing amounts of data delivered to Big Data analysis platforms and tools. This demand is driven by the desire to better understand user experience, quality of service, real time performance of systems, and the undiscovered patterns and opportunities that exist to improve service, sell products, and delight users. Capital One Technology solved for the ever increasing demand for data from consumer facing applications using Spring Extreme Data (XD) and custom Java libraries to stream data in real time from applications to Hadoop HDFS, Mongo, Kafka, Splunk, and others. In this talk we will discuss our journey from batch-oriented processes using databases to a real time data streaming solution and the significant benefits achieved. We’ll cover the implications of adopting a streaming solution, why we selected Spring XD, and the target architecture we are implementing to land data in HDFS. We’ll cover how we automated environment provisioning, the design for our client libraries (Java, Spring), our XD environment, and how we’re tapping into all that data. We’ll describe challenges we overcame on this journey including connecting to our HDFS cluster, working with multiple Mongo stores, using Kerberos, and ensuring protection and encryption of sensitive data end to end. Lastly we’ll talk about a number of use cases where we are evaluating Spring XD and see potential benefits including “rolling window” system analytics, digital message delivery, and event driven architectures.

Get the Spring newsletter

Stay connected with the Spring newsletter

Subscribe

Get ahead

VMware offers training and certification to turbo-charge your progress.

Learn more

Get support

Tanzu Spring offers support and binaries for OpenJDK™, Spring, and Apache Tomcat® in one simple subscription.

Learn more

Upcoming events

Check out all the upcoming events in the Spring community.

View all