Reactive Summit 2017

This talk discusses techniques for achieving optimized performance, availability, cost or other attributes of a distributed system. Firstly, the presentation introduces and in depth explains optimization techniques used in state of the art large scale stream and fast data processing frameworks such as Akka Streams, Spark or Flink, including logical and physical optimizations or code generation. Consequently, powerful optimization concepts applicable to general distributed systems, including systems built using Akka, are explained on examples. Finally, the presentation highlights the role of machine learning and artificial intelligence in the area and explains how machine generated data such as logs and metrics can be used to model, minimize, maximize or find the perfect balance of selected attributes of the system, demonstrated on examples from practice. The attendees will gain an understanding of the available optimization approaches, tradeoffs and the value of machine learning and intelligence and ultimately will be able to apply some of the techniques to optimize general distributed systems as well as streaming data processing systems built using Spark, Flink or Akka Streams.