Limit(0)

2024년 7월 5일 금요일

Producers Acks Deep Dive acks = all (replicas acks)

1. Leader + Replicas ack requested

2. Added latency and safety

3. No data loss if enough replicas

- Necessary setting if you don't want to lose data

- Acks=all must be used in conjunction with min.insync.replicas.

- min.insync.replicas can be set at the broker or topic level (override).

- min.insync.replicas=2 implies that at least 2 brokers that are ISR(including leader) must responsd that they have the data.

- That means if you use replication.factor=3, min.insync=2, acks=all, you can only tolerate I broker going down, otherwise the producer will receive an exception on send.

Producers Acks Deep Dive acks = 1 (leader acks)

1. Leader response is requested, but replication is not a guarantee

(happens in the background)

2. If an ack is not received, the produceder may retry

3. If the leader broker goes offline but replicas haven't replicated the data yet, we have a data loss.

Producers Acks Deep Dive acks = 0 (no acks)

1. No response is requested

2. If the broker goes offline or an exception happens, we won't know and will lose data

3. Useful for data where it's okay to potentially lose messages:

- Metrics collection

- Log collection

2024년 6월 22일 토요일

Kafka Command Line Interface 101

kafka-topics

C:\kafka_2.12-3.7.0>.\bin\windows\kafka-topics.bat --bootstrap-server localhost:9092,localhost:9093,localhost:9094 --create --topic mytopic --partitions 2 --replication-factor 1

Created topic mytopic.