페이지

2024년 9월 22일 일요일

Advanced Kafka Configuration Important parameters to be aware of

1. auto.create.topics.enable=true => set to false in production

2. background.threads=10 => increase if you have a good CPU

3. delete.topic.enable=false => your choice

4. log.flush.interval.messages => don't ever change. Let your OS do it

5. log.retention.hours=168 => Set in regards to your requirements

6. message.max.bytes=1000012 => increase if you need more than 1MB

7. min.insync.replicas=1 => set to 2 fi you want to be extra safe

8. num.ios.threads=8 => ++if your network io is a bottleneck

9. num.network.threads=3 => ++if your network is a bottleneck

10. num.recovery.threads.per.data.dir=1 => set to number of disks

11. num.replica.fetchers=1 => increase if your replicas are lagging

12. offsets.retention.minutes=1440 => after 24 hours you lose offsets

13. unclean.leader.election.enable=true => false if you don't want data loss

14. zookeeper.session.timeout.ms=6000 => increase if you timeout often

15. broker.rack=null => set your to availability zone in AWS

16. default.replication.factor=1 => set to 2 or 3 in a kafka cluster

17. num.partitions=1 => set from 3 to 6 in your cluster

18. quota.producer.default=10485760 => set quota to 10MBs

19. quota.consumer.default=10485760 => set quota to 10MBs


댓글 없음: