Skip to content
Codeloom

Topics / Apache Kafka

Apache Kafka

Distributed event streaming platform. Topics, producers, consumers, and real-time data pipelines.

Browse 21 tutorials → Other topics

Why learn Apache Kafka?

  • The backbone of real-time data infrastructure at scale.

  • Used by LinkedIn, Netflix, Uber, and most large-scale data platforms.

  • Handles millions of events per second with fault tolerance and durability.

  • Essential for event-driven architectures and stream processing.

What you can build with Apache Kafka

Real-time event streaming Log aggregation and metrics collection Event-driven microservices Change data capture (CDC) Stream processing with Kafka Streams

Apache Kafka tutorials

21 articles · page 1 of 2

Hand-written tutorials, ordered as a recommended learning path.

  1. 01 What Is Kafka? A practical introduction to Apache Kafka — what it is, why it exists, its core concepts, and how it differs from traditional message queues. Includes your first producer and consumer code.
  2. 02 Architecture A detailed look at Kafka's internal architecture — broker clusters, KRaft consensus, replication protocols, partition leadership, log segments, and how they combine to deliver fault tolerance at scale.
  3. 03 Installation & Setup Step-by-step guide to running Apache Kafka locally using KRaft mode binaries and Docker Compose — includes Schema Registry, Kafka UI, topic creation, and CLI producer/consumer testing.
  4. 04 Topics, Partitions & Offsets A deep dive into Kafka's core data model — how topics organize data, how partitions enable parallelism and ordering, how offsets track consumer progress, and how retention and compaction manage storage.
  5. 05 Producers Learn how Kafka producers work including batching, partitioning, serialization, delivery guarantees, and idempotent production with Python and Java examples.
  6. 06 Consumers & Groups Master Kafka consumers and consumer groups including poll loops, offset management, rebalancing strategies, and partition assignment with Python and Java examples.
  7. 07 Kafka Streams Learn Kafka Streams for real-time data processing including KStream, KTable, windowed aggregations, joins, exactly-once semantics, and state stores with Java examples.
  8. 08 Kafka Connect Master Kafka Connect for integrating external systems with Kafka using source and sink connectors, Debezium CDC, JDBC, Elasticsearch, REST API management, and SMTs.
  9. 09 Schema Registry & Avro Learn how Confluent Schema Registry manages Avro schemas for Kafka producers and consumers, enabling safe schema evolution and decoupled services.
  10. 10 Security Configure SSL/TLS encryption, SASL authentication with SCRAM-SHA-256, and ACL authorization to secure your Apache Kafka cluster in production.
  11. 11 Performance Tuning Master Apache Kafka performance tuning with partition strategies, producer and consumer optimization, compression trade-offs, and real-world architecture patterns.
  12. 12 Exactly-Once Semantics Understand Kafka's three delivery guarantees, idempotent producers, transactional writes, and when you actually need exactly-once semantics in production.
  13. 13 Event-Driven Architecture Master event-driven architecture patterns including event sourcing, CQRS, and the saga pattern using Apache Kafka for real-world microservices systems.
  14. 14 Monitoring & Operations Learn the essential Kafka metrics to monitor, alerting strategies, operational tools like Prometheus and Grafana, and capacity planning for production clusters.
  15. 15 Building Data Pipelines Build production-grade data pipelines using Kafka Connect, Debezium CDC, sink connectors, schema evolution, and dead letter queues for robust error handling.
  16. 16 KRaft & Cluster Mgmt Understand why Kafka is removing ZooKeeper, how KRaft consensus works, and master cluster operations including broker management and disaster recovery.
  17. 17 Real-World Use Cases Explore how Netflix, Uber, LinkedIn, and others use Apache Kafka in production for recommendations, pricing, fraud detection, IoT, and log aggregation.
  18. 18 Dead Letter Queues Learn how to implement Dead Letter Queues in Kafka to gracefully handle failed messages with retry strategies, monitoring, and production patterns.
  19. 19 Multi-Cluster Replication Master Kafka multi-cluster replication using MirrorMaker 2 with active-passive and active-active topologies, offset sync, and disaster recovery.
  20. 20 Message Queues Compare RabbitMQ, Apache Kafka, and Amazon SQS with working code examples, architecture diagrams, and decision criteria for choosing the right message queue.