페이지

2024년 7월 5일 금요일

Idempotent Producer

 1. Here's the problem: the Producer can introduce duplicate messages in Kakka due to network errors

- In Kafka >= 0.11, you can define a "idempotent producer" which won't introduce duplicates on network error


- Idempotent producers are great to guarantee a stable and safe pipeline!
- They come with:
    - retries = Integer.MAX_VALUE(2^31-1 = 2147483647)
    - max.in.flight.requests = I (Kafka >= 0.11 & < 1.1) or
    - max.in.flight.requests = 5 (Kafka >= 1.1 - higher performance)
    - acks = all

- Just set:
    - producerProps.put("enable.idempotence", true);












댓글 없음: