Spring Boot and Kafka quickstarts

Producer • Consumer • Streams

Code samples around Kafka Clients and Kafka Streams leveraging Spring Boot to simplify the development of applications.

Requirements

• Java 21
• Maven
• Docker

Quickstarts list

Producers

Module	Library	Content
Avro Generic	Kafka Clients	Produce generic Avro records
Avro Specific	Kafka Clients	Produce specific Avro records
Headers	Kafka Clients	Produce records with headers
Simple	Kafka Clients	Produce String records
Transaction	Kafka Clients	Produce to multiple topics while guaranteeing atomicity

Consumers

Module	Library	Content
Avro Generic	Kafka Clients	Consume generic Avro records
Avro Specific	Kafka Clients	Consume specific Avro records
Circuit breaker	Kafka Clients	Consume records while handling poison pills
Headers	Kafka Clients	Consume records with headers
Retry external system	Kafka Clients	Consume records while retrying on failed external system calls
Simple	Kafka Clients	Consume String records
Transaction	Kafka Clients	Consume records from committed transactions

Kafka Streams

Source

Module	Library		DSL
Global Table	Kafka Streams	Source topic as global table	globalTable()
Table	Kafka Streams	Source topic as table	table()
Stream	Kafka Streams	Source topic as stream	stream()

Stateless

Module	Library	Content	DSL
Branch	Kafka Streams	Split and create branches from a stream	split(), branch()
Cogroup	Kafka Streams	Aggregate records of multiple streams by key	cogroup()
Filter	Kafka Streams	Retain or drop records based on a predicate	filter(), filterNot()
FlatMap	Kafka Streams	Change one record into 0, 1 or n records	flatMap()
FlatMapValues	Kafka Streams	Change one record value into 0, 1 or n record values	flatMapValues()
Foreach	Kafka Streams	Perform a terminal operation on each record	foreach()
Map	Kafka Streams	Change one record into another record	map()
MapValues	Kafka Streams	Change one record value into another record value	mapValues()
Merge	Kafka Streams	Merge two streams into one stream	merge()
Print	Kafka Streams	Print a stream to the system output or a file	print()
Repartition	Kafka Streams	Trigger a repartitioning of the stream	repartition()
SelectKey	Kafka Streams	Change the key of each record	selectKey()

Aggregate

Module	Library	Content	DSL
Aggregate	Kafka Streams	Aggregate a stream by key in a single object	groupByKey(), aggregate()
Aggregate Hopping Window	Kafka Streams	Aggregate a stream by key and by hopping window with a grace period	groupByKey(), aggregate(), windowedBy(), advanceBy()
Aggregate Sliding Window	Kafka Streams	Aggregate a stream by key and by sliding window with a grace period	groupByKey(), aggregate(), windowedBy()
Aggregate Tumbling Window	Kafka Streams	Aggregate a stream by key and by tumbling window with a grace period	groupByKey(), aggregate(), windowedBy()
Average	Kafka Streams	Compute an average value of a stream by key	groupBy(), aggregate()
Count	Kafka Streams	Count the number of records of a stream by key	groupBy(), count()
Reduce	Kafka Streams	Reduce the records of a stream by key	groupBy(), reduce()

Join

Module	Library	Content	DSL
Join Stream-Global Table	Kafka Streams	Perform an inner join between a stream and a global table	join()
Join Stream-Stream	Kafka Streams	Perform an inner join between two streams	join()
Join Stream-Table	Kafka Streams	Perform an inner join between a stream and a table	join()
Left Join Stream-Global Table	Kafka Streams	Perform a left join between a stream and a global table	leftJoin()
Left Join Stream-Stream	Kafka Streams	Perform a left join between two streams	leftJoin()
Left Join Stream-Table	Kafka Streams	Perform a left join between a stream and a table	leftJoin()
Outer Join Stream-Stream	Kafka Streams	Perform an outer join between two streams	outerJoin()

Windowing

Module	Library	Content	DSL
Hopping Window	Kafka Streams	Group records by hopping window with a grace period	windowedBy(), advanceBy()
Sliding Window	Kafka Streams	Group records by sliding window with a grace period	windowedBy()
Tumbling Window	Kafka Streams	Group records by tumbling window with a grace period	windowedBy()

Processor

Module	Library	Content	DSL	Processor API
Process	Kafka Streams	Apply a processor to a stream	process()	context(), forward(), Record#headers()
ProcessValues	Kafka Streams	Apply a fixed key processor to a stream	processValues()	context(), forward(), Record#headers()
Schedule	Kafka Streams	Schedule punctuation functions based on wall clock time and stream time	process()	schedule(), getStateStore()

Stores

Module	Library	Content	DSL	Processor API
Store Cleanup	Kafka Streams	Schedule periodic store cleanup based on stream time	process()	schedule(), getStateStore()
Key-Value Store	Kafka Streams	Insert and retrieve records from a key-value store	process(), addStateStore()	getStateStore()
Timestamped Key-Value Store	Kafka Streams	Insert and retrieve records from a timestamped key-value store	process(), addStateStore()	getStateStore()
Window Store	Kafka Streams	Insert and retrieve records from a window store	process(), addStateStore()	getStateStore()
Timestamped Window Store	Kafka Streams	Insert and retrieve records from a timestamped window store	process(), addStateStore()	getStateStore()

Handler

Module	Library	Content	Config
Deserialization Exception Handler	Kafka Streams	Handle deserialization exceptions	default.deserialization.exception.handler
Processing Exception Handler	Kafka Streams	Handle processing exceptions	processing.exception.handler
Production Exception Handler	Kafka Streams	Handle production and serialization exceptions	default.production.exception.handler

Credits

Confluent - Crossing the Streams

Bill Bejeck - Windowing time semantics

Apache Kafka - Sliding Windows

Jeanlouisboudart - Retriable Consumer