Kafka Developer
- P2P SOFTTEK Staffing Services
1 Days Ago
NA
C2C
Strongsville-OH, Pittsburgh-PA, Dallas-TX
11-15 Years
Required Skills: Confluent Kafka, Apache Kafka, Confluent Flink, Apache Flink, Real-time architecture, Java, Python, Kafka, Flink, AKS, EKS, Azure, AWS, CI/CD pipelines
Job Description
Overall Experience
- 10+ years of experience in building real-time, event-driven applications using Java/Python, Kafka, and Flink
- Strong experience with containerization and deployments on AKS/EKS (Azure/AWS)
- Proven expertise in building scalable, high-volume streaming solutions using Kafka and Flink
- Experience in modernizing legacy integrations to real-time streaming architectures
- Hands-on experience in building CI/CD pipelines for Kafka and Flink deployments
1. Confluent Kafka / Apache Kafka
- Kafka topic design and partition management
- Producer/consumer configuration and optimization
- Schema Registry setup and governance
- Kafka broker setup, scaling, and load balancing
- Kafka Connect configuration and troubleshooting
2. Confluent Flink / Apache Flink
- Flink JobManager and TaskManager setup and configuration
- Stateful stream processing and checkpointing
- State management and persistence for intraday processing
- Real-time aggregations and windowing
- Scaling and performance tuning of Flink jobs
- Batch and streaming unification
3. Real-Time Architecture Design
- Event-driven architecture design and best practices
- Data mesh concepts and implementation
- Kafka and Flink integration patterns
- Multi-source data ingestion and aggregation into Kafka topics
- Downstream consumer and API integration design
