- Consuemers choose when to commit offsets.
- There are 3 delivery sematics:
- At most once:
- offsets are committed as soon as the message is received.
- If the processing goes wrong, the message will be lost(it won't be read again).
- At least once(usually preferred):
- offsets are committed after the message is processed.
- If the processing goes wrong, the message will be read again.
- This can result in duplicate processing of messages. Make sure your processing is idempotent (i.e. processing again the messages won't impact your systems)
- Exactly once:
- Can be achieved for Kafka => Kafka workflows using Kafka Streams API
- For Kafka => External System workflows, use an idempotent consumer.
댓글 없음:
댓글 쓰기