Introduction to Streaming Systems- Fast data
- Streaming architecture
- Lambda architecture
- Message queues
- Streaming processors
Introduction to Kafka- Architecture
- Comparing Kafka with other queue systems (JMS / MQ)
- Kaka concepts : Messages, Topics, Partitions, Brokers, Producers, commit logs
- Kafka & Zookeeper
- Producing messages
- Consuming messages (Consumers, Consumer Groups)
- Message retention
- Scaling Kafka
Programming With Kafka- Configuration parameters
- Producer API (Sending messages to Kafka)
- Consumer API (consuming messages from Kafka)
- Commits, Offsets, Seeking
- Schema with Avro
Kafka Streams- Streams overview and architecture
- Streams use cases and comparison with other platforms
- Learning Kafka Streaming concepts (KStream, KTable, KStore)
- KStreaming operations (transformations, filters, joins, aggregations)
Administering Kafka- Hardware / Software requirements
- Deploying Kafka
- Configuration of brokers / topics / partitions / producers / consumers
- Security: How secure Kafka cluster, and secure client communications (SASL, Kerberos)
- Monitoring : monitoring tools
- Capacity Planning : estimating usage and demand
- Trouble shooting : failure scenarios and recovery
Monitoring and Instrumenting Kafka- Monitoring Kafka
- Instrumenting with Metrics library
- Instrument Kafka applications and monitor their performance
, Introduction to Streaming Systems
Understanding Fast data
Streaming terminologies
Understanding at-least-once / at-most-once / exactly-once processing patterns
Popular streaming architectures
Lambda architecture
Streaming platforms overview
Lab: Hands-on first look at Kafka
Introducing Kafka
Comparing Kafka with other queue systems (JMS / MQ)
Kafka Architecture
Kaka concepts: Messages, commit logs
Kafka & Zookeeper
Producing messages
Consuming messages
Consumers, Consumer Groups
Message retention
Scaling Kafka
Kafka across multiple data centers and disaster recovery
Lab: Getting Kafka up and running
Lab: Using Kafka utilities
Using Kafka APIs
Configuration parameters
Producer API - sending messages to Kafka
Consumer API - consuming messages from Kafka
Producer send modes
Message compression
Commits, Seeking
Managing offsets - auto commit / manual commit
Lab: Writing Producer / Consumer
Lab: Benchmarking Producer send modes
Lab: Comparing compression schemes
Lab: Managing offsets
Lab: Clickstream processing
Kafka Streams API
Introduction to Kafka Streams library
Features and design
Streams concepts: KStream / KTable / KStore
Streaming operations (transformations, aggregations)
Using Streams API: foreach / filter / map / groupby
Lab: Kafka Streaming APIs
Monitoring & Troubleshooting Kafka
Monitoring tools overview
Monitoring Kafka
Cluster level and host level monitoring
Identifying performance bottlenecks
Troubleshooting common Kafka issues
Bonus Content / Time Permitting
Kafka Best Practices
Avoiding common mistakes
Hardware selection
Cluster sizing
Partition sizing
Zookeeper settings
Compression and batching
Message sizing
Monitoring and instrumenting
Troubleshooting
Download Overview