페이지

2024년 8월 30일 금요일

Kafka Cluster Size Discussions 100 brokers

 


1. In the case of 100 brokers:

    - Your cluster is fully distributed and can handle tremendous volumes, even using commodity hardware.

    - Zookeeper may be under pressure because of many open connections, so you need to increase the Zookeeper instance performance

    - Cluster management is a full time job (make suer no broker acts weirdly)

    - It is not recommended to have a replication factor of 4 or more, and this would incur network communications within the brokers. Leave it as 3

    - Scale horizontally only when a bottleneck is reached (network, disk i/o, cpu, ram)

댓글 없음: