Kafka Connect & Data Integration Training
Build powerful data integration pipelines with this focused 2-day training on Kafka Connect. Learn to deploy and manage source and sink connectors, implement change data capture with Debezium, apply Single Message Transforms, and operate Connect clusters in production.
Training Details
Section titled “Training Details”| Duration | 2 days (16 hours) |
| Level | Intermediate |
| Delivery | In-person, Live online, Hybrid |
| Certification | N/A |
Who Is This For?
Section titled “Who Is This For?”- Data engineers building data integration pipelines
- Backend developers connecting Kafka to databases and services
- Platform engineers managing Kafka Connect infrastructure
- Anyone building CDC pipelines or data lake ingestion
Learning Outcomes
Section titled “Learning Outcomes”After completing this training, you’ll be able to:
- Deploy and manage Kafka Connect in distributed mode
- Configure source connectors for database CDC with Debezium
- Set up sink connectors for Elasticsearch, S3, and databases
- Apply Single Message Transforms for data transformation
- Monitor connector health and handle failures
- Design production data integration architectures
Detailed Agenda
Section titled “Detailed Agenda”Day 1: Kafka Connect and Source Connectors
Section titled “Day 1: Kafka Connect and Source Connectors”Module 1: Kafka Connect Architecture
- Standalone vs distributed mode
- Workers, tasks, and converters
- REST API for connector management
- Plugin installation and class loading
- Hands-on: Deploy a distributed Connect cluster
Module 2: Debezium CDC Connectors
- Change data capture concepts
- PostgreSQL, MySQL, and MongoDB connectors
- Snapshot modes and initial data loading
- Schema change handling
- Hands-on: Set up CDC from PostgreSQL to Kafka
Module 3: Source Connector Patterns
- JDBC source connector for legacy databases
- File and S3 source connectors
- HTTP source for REST APIs
- Custom source connector development basics
- Hands-on: Build a multi-source data ingestion pipeline
Day 2: Sink Connectors and Production
Section titled “Day 2: Sink Connectors and Production”Module 4: Sink Connectors
- Elasticsearch sink for search indexing
- S3/GCS sink for data lake ingestion (Parquet, Avro)
- JDBC sink for database synchronization
- MongoDB and Redis sink connectors
- Hands-on: Build a CDC-to-Elasticsearch pipeline
Module 5: Transforms and Schema Registry
- Single Message Transforms (SMTs)
- Built-in transforms — route, extract, flatten, timestamp
- Chaining transforms for complex processing
- Schema Registry integration with converters
- Hands-on: Transform and route messages with SMTs
Module 6: Production Operations
- Connector monitoring with JMX and Prometheus
- Error handling — dead letter queues, error tolerance
- Exactly-once delivery with sink connectors
- Scaling and performance tuning
- Hands-on: Configure monitoring, alerting, and error handling
What’s Included
Section titled “What’s Included”- Access to hands-on lab environments with databases and Kafka
- Course slides and reference materials
- Connector configuration templates
- Post-training email support (30 days)
Request This Training
Section titled “Request This Training”Ready to bring Kafka Connect training to your team? Contact me to discuss dates, group size, and customization options.