Kafka动态调整topic分区partition

Kafka动态调整topic分区partition在使用 kafka 时 初期创建 topic 时所指定的 topic 属性有时会需要修改 如何动态修改 kafka topic 属性 kafka 提供了命令行工具 kafka topics sh kafka topics sh 工具介绍 kafka topics sh 工具也是我们用来创建 topic 查看 topic 详情的工具 直接运行 kafka topics sh 可以看出 它是用来创建 删除

在使用kafka时,初期创建topic时所指定的topic属性有时会需要修改,如何动态修改kafka topic属性?kafka提供了命令行工具—kafka-topics.sh.

kafka-topics.sh工具介绍

kafka-topics.sh工具也是我们用来创建topic、查看topic详情的工具。
直接运行kafka-topics.sh可以看出,它是用来创建、删除、查看以及更新topic的
root@ubuntu:/opt/kafka_2.11-1.1.0/bin# ./kafka-topics.sh
Create, delete, describe, or change a topic.
Option Description

----

–alter Alter the number of partitions,
replica assignment, and/or
configuration for the topic.
–config A topic configuration override for

更新或者修改topic
注意:我的kafka版本是1.1.0, 并且我只有一个broker。
1, 首先我们创建一个topic,然后查看详情
root@ubuntu:/opt/kafka_2.11-1.1.0/bin# ./kafka-topics.sh –create –zookeeper 192.168.119.131:2181 –replication-factor 1 –partitions 4 –topic yqtopic1
Created topic “yqtopic1”.
root@ubuntu:/opt/kafka_2.11-1.1.0/bin#

root@ubuntu:/opt/kafka_2.11-1.1.0/bin# ./kafka-topics.sh –describe –zookeeper 192.168.119.131:2181 –topic yqtopic1
Topic:yqtopic1 PartitionCount:4 ReplicationFactor:1 Configs:
Topic: yqtopic1 Partition: 0 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 1 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 2 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 3 Leader: 0 Replicas: 0 Isr: 0
root@ubuntu:/opt/kafka_2.11-1.1.0/bin#

2,修改刚创建的topic,并查看修改的情况
将分区数有4修改为12
root@ubuntu:/opt/kafka_2.11-1.1.0/bin# ./kafka-topics.sh –alter –zookeeper 192.168.119.131:2181 –topic yqtopic1 –partitions 12
WARNING: If partitions are increased for a topic that has a key, the partition logic or ordering of the messages will be affected
Adding partitions succeeded!
root@ubuntu:/opt/kafka_2.11-1.1.0/bin#

root@ubuntu:/opt/kafka_2.11-1.1.0/bin# ./kafka-topics.sh –describe –zookeeper 192.168.119.131:2181 –topic yqtopic1 Topic:yqtopic1 PartitionCount:12 ReplicationFactor:1 Configs:
Topic: yqtopic1 Partition: 0 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 1 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 2 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 3 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 4 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 5 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 6 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 7 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 8 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 9 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 10 Leader: 0 Replicas: 0 Isr: 0
Topic: yqtopic1 Partition: 11 Leader: 0 Replicas: 0 Isr: 0
root@ubuntu:/opt/kafka_2.11-1.1.0/bin# ls -al /tmp/kafka-logs/
total 72
drwxr-xr-x 14 root root 4096 Oct 13 14:34 .
drwxrwxrwt 17 root root 4096 Oct 13 14:34 …
-rw-r–r– 1 root root 0 Oct 13 14:10 cleaner-offset-checkpoint
-rw-r–r– 1 root root 0 Oct 13 14:10 .lock
-rw-r–r– 1 root root 4 Oct 13 14:33 log-start-offset-checkpoint
-rw-r–r– 1 root root 54 Oct 13 14:10 meta.properties
-rw-r–r– 1 root root 163 Oct 13 14:33 recovery-point-offset-checkpoint
-rw-r–r– 1 root root 163 Oct 13 14:34 replication-offset-checkpoint
drwxr-xr-x 2 root root 4096 Oct 13 14:20 yqtopic1-0
drwxr-xr-x 2 root root 4096 Oct 13 14:20 yqtopic1-1
drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-10
drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-11
drwxr-xr-x 2 root root 4096 Oct 13 14:20 yqtopic1-2
drwxr-xr-x 2 root root 4096 Oct 13 14:20 yqtopic1-3
drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-4
drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-5
drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-6
drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-7
drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-8
drwxr-xr-x 2 root root 4096 Oct 13 14:33 yqtopic1-9
root@ubuntu:/opt/kafka_2.11-1.1.0/bin#

修改后的截图如下

编程小号
上一篇 2025-04-08 14:01
下一篇 2025-01-23 22:11

相关推荐

版权声明:本文内容由互联网用户自发贡献,该文观点仅代表作者本人。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌侵权/违法违规的内容, 请发送邮件至 举报,一经查实,本站将立刻删除。
如需转载请保留出处:https://bianchenghao.cn/hz/144430.html