1. Kafka can only operate well in a single resion
2. Therefore, it is very common for enterprises to have Kafka clusters across the world, with some level of replication between them
3. A replication application at its core is just a consumer + a producer
4. There are different tools to perform it:
- Mirror Maker - open source tool that ships with Kafka
- netflix users Flink - they wrote their own applicaiton
- Uber uysers uRepli8cator - address performance and operations issues with MM
- Comcast has their own open source Kafka Connect Source
- Confluent has their own Kafka Connect Source(paid)
5. Overall, try these and see if it works for your use case before writing your own
6. There are two desings for cluster replication:
7. Active => Active:
- You have a global application
- You have a global dataset
8. Active => Passive:
- You want to have an aggregation cluster (for example for analytics)
- You want to create some form of disaster recovery strategy (it's hard)
- Cloud Migration (from on-premise cluster to Cloud cluster)
9. Replicating doesn't preserve offsets, just data!
댓글 없음:
댓글 쓰기