Apache Kafka Foundation

Introduction
In this training, candidates are expected to have experience with Apache Kafka and they have programmed Kafka consumers and producers before. Note: All examples and exercises are programmed with Spring Kafka. We will practice with messaging delivery guarantees and investigate ordering in Kafka. We also learn how to detect, prevent and handle errors that can occur when sending messages with Kafka. And learn the basics of Kafka Streams.

Detailed description
This training starts with a short review of what producers and consumers are and their most important properties. Then we will continue to discuss Kafka’s delivery guarantees: at most once, at least once, exactly once and look at message ordering within Kafka. We will do an exercise in which we program the producer and consumer so that a message arrives exactly once and in the order in which the message was sent. After that we will discuss best practices with rebalancing partitions. Next topic of the day is error management. We will look at which kind of errors can occur and what you can do when errors occur. We practice different retry strategies in Spring Kafka Consumer. We will also look at which Kafka metrics are handy to monitor and how to trace messages. Last topic of the day is Kafka streams. We will learn the basic concepts of stream processing and look at the Kafka streaming API. As exercise we will create a Kafka Stream Application.

Target audience
The training is intended for Java developers who have already followed the Kafka Essentials course or already have a lot of experience working with Kafka. This is a hands-on training aimed at developers. A lot of attention is paid to creating and studying code examples of Kafka Clients that you can use when working on a project with Apache Kafka.

Learning goals

• Know about and practice with Kafka’s delivery and ordering guarantees
• How to prevent problems with rebalancing partitions
• Detect, handle and prevent errors in producers and consumers
• Create a Kafka Streams application

Skills acquired

• Knowledge about Kafka delivery and ordering guarantees
• Program and configure Spring Kafka clients such that a message is delivered exactly once in a guaranteed order
• Best practices rebalancing partitions
• How to design for errors in producers and consumers
• Know which Kafka metrics are handy to monitor
• Knowledge about and programming experience with Kafka Streams

Topics

• Short summary what are the important characteristics of producers and consumers
• Delivery guarantees: at most once, at least once, exactly once
• Ordering guarantees: in what order are messages processed
• Rebalancing
• Error management
• Monitoring and tracing messages
• Kafka Streams

Training outline

Day 1: Our Apache Kafka Essentials training.
Day 2:

Kafka Basics: Producers, Consumers: +/- 0.5h

• Brief review of what producers and consumers are and their most important properties
• Setup local environment for exercises

Delivery Semantics: +/- 2h

• Discuss delivery guarantees Producer
• Discuss ordering guarantees Producer
• Programming exercises about delivery and ordering guarantees with Spring Kafka Producer
• Discuss delivery guarantees Consumer
• Discuss ordering guarantees Consumer
• Programming exercises about delivery and ordering guarantees with Spring Kafka Consumer

Rebalancing +/- 0.5h

• Consumer groups and the rebalance protocol
• Pitfalls

Error Management: +/- 1,5h

• Producer errors and retry strategies
• Consumer errors and retry strategies
• Dead letter topic
• Programming exercises about retry strategies and dead letter topics with Spring Kafka Consumer
• Monitoring and Tracing messages

Kafka Streams: +/ 3h

• Introduction Stream Processing
• Kafka Stream API
• Stateless vs Stateful streaming
• K tables
• Windowing

Prerequisites

• Experience with Java programming
• Experience with programming Kafka Producers and Consumers
• Basic knowledge of Apache Kafka
• Good to have experience with Maven, Docker and Spring Boot

Provided training material
After doing the training you will have good code examples that you can use when starting an Apache Kafka project. After the training you will receive possible solutions of the exercises and the slides in PDF format.

About the trainer
When Mireille was a little girl, she wanted to be an inventor or a surgeon or a school teacher. But now that she’s been a software developer for several years, she doesn’t want to be anything else. She likes to build beautiful applications and solve complex problems. She also likes to know how things work, and to share that knowledge when she knows. Mireille has worked with Apache Kafka on several projects and is very excited about this messaging and event streaming platform. She is looking forward to teach you more about Kafka.

Note: This training can be given in Dutch or English at one of the OpenValue offices (Utrecht, Amsterdam, Rotterdam, Arnhem, Munich, Dusseldorf, Vienna, Zurich) or at your own location. Please contact us to discuss possibilities for a remote training and for training in German.