Some useful commands to manage Kafka

This post shows some useful commands to manage Kafka. (based on version 2.12)

Detailed information is on Kafka official documentation

Notice that some commands use zookeeper ip/port and others use kafka ip/port

Topic management commands

List all topics

${KAFKA_HOME}/bin/kafka-topics.sh –list –zookeeper zookeeper_ip:port

Create a topic

${KAFKA_HOME}/bin/kafka-topics.sh –create –zookeeper zookeeper_ip:port –replication-factor num –partitions num –topic topic_name

Get information for a topic

${KAFKA_HOME}/bin/kafka-topics.sh –describe –zookeeper zookeeper_ip:port –topic topic_name

Delete a topic

${KAFKA_HOME}/bin/kafka-topics.sh –delete –zookeeper zookeeper_ip:port –topic topic_name

Get offset for a topic

${KAFKA_HOME}/bin/kafka-run-class.sh kafka.tools.GetOffsetShell –broker-list kafka_ip/port –topic topic_name –time time_option

  • time_option : -1 (latest), -2 (earliest)

Consumer management commands

List all consumer groups registered on Kafka

${KAFKA_HOME}/bin/kafka-consumer-groups.sh –bootstrap-server kafka_ip/port –list

Get information for a consumer group

${KAFKA_HOME}/bin/kafka-run-class.sh kafka.admin.ConsumerGroupCommand –bootstrap-server kafka_ip/port –group group_name –describe

Reset offset for a consumer group

${KAFKA_HOME}/bin/kafka-run-class.sh kafka.admin.ConsumerGroupCommand –bootstrap-server kafka_ip/port –topic topic_name –group group_name –reset-offsets offset_option –execute

  • offset_option : –to-earliest | –to-latest | –to-offset [offset_number]
  • –to-earliest : the first offset after the consumer’s last committed offset, that is, not the beginning of the partition
  • –to-latest : the latest offset of the partition
  • –to-offset [offset_number] : offset number must be bigger thatn earliest offset

Delete a consumer group from the topic

${KAFKA_HOME}/bin/kafka-run-class.sh kafka.admin.ConsumerGroupCommand –bootstrap-server kafka_ip/port –group group_name –delete

 

 

 

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.