Loading…
This event has ended. Visit the official site or create your own event on Sched.
Back To Schedule
Thursday, October 19 • 10:00am - 10:50am
Akka Streams to the Rescue

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Tweet Share
In this talk, you will learn about the challenges we faced at Credit Karma in our Data Transformation project and the solutions we ultimately arrived at, including:
- The way we transformed several billions of data points stored in MySQL, which gets live updates
- How we ran large volume, CPU intensive data transformations by adapting to available processing headroom without affecting the live load on MySQL. How Akka Streams simplified our implementation of high throughput data flow and processing - with optimized CPU and memory usage. We will also touch on some challenges we faced with Akka Streams.
- Akka Streams don’t cross JVM process boundaries. So, how did we solve this challenge in a distributed environment?
- Finally, we’ll look at some performance numbers on our approach.
You will walk away with implementation ideas and performance numbers to help them design large scale data processing/transformation projects based on Akka streams.
Speakers
avatar for Yubin Liang

Yubin Liang

Staff Software Engineer, CREDIT KARMA
Yubin is a staff software engineer at Credit Karma. He works on the core services, which support requests from Credit Karma’s website, mobile apps, email systems and background jobs. He leads projects that process huge volumes of data for 75 million members at scale. Before Credit... Read More →
avatar for Sajeev Pillai

Sajeev Pillai

Senior Engineering Manager, Credit Karma, Inc
Sajeev is a Senior Engineering Manager with Credit Karma's Core Services team. He focuses on architecting and delivering security solutions that serve Credit Karma's 75 million members. Sajeev is passionate about problems that cross both security and platforms, and has worked in various... Read More →

Thursday October 19, 2017 10:00am - 10:50am CDT
Salon F/G
  session

Attendees (67)