How to delete multiple topics in Apache Kafka

kafka delete all topics
delete messages from kafka topic java
msk delete topic
kafkacat delete --topic
kafka get retention time for topic
confluent control center delete topic
kafka list topics
delete'' retention ms kafka

Assuming that I have a number of topics with the same prefix, e.g:

giorgos-topic1
giorgos-topic2
giorgos-topic3
...

The command used for deleting a single topic (say giorgos-topic1) is the following:

./bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic giorgos-topic1

Is it possible to delete multiple columns using a single command and possibly a regular expression/wildcard (e.g. giorgos-*) instead of typing all the topic names that need to be deleted one by one?

Yes you can use regex-like expressions when deleting topics with the kafka-topics.sh tool:

For example, to delete all topics starting with giorgos-:

./bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic 'giorgos-.*'

Using the Admin APIs, you can also delete several topics at once, see AdminClient.deleteTopics

How can I delete all the messages from a Kafka topic?, Start a Kafka broker with broker ID 100 . Create remove-me topic. $ ./bin/kafka-topics.sh --zookeeper localhost:2181 \ --create \ --topic remove-me \ --partitions 1 \ --replication-factor 1 Created topic "remove-me". Topic deletion is enabled by default in new Kafka versions ( from 1.0.0 and above). If you are using older versions of Kafka, you have to change the configuration of broker delete.topic.enable to true (by default false in older versions) These are some basics of Kafka topics. In the next article, we will look into Kafka producers.

In cases where regex is not possible we can use a comma seperated list of topic names for the deletion of topics.

kafka-topics.sh --zookeeper 10.0.0.160:2181 --delete --topic giorgos-topic1,giorgos-topic2,giorgos-topic3,...

Delete a topic, delete all kafka topics on a given cluster. kafka-del-topics.sh. #!/bin/bash. TOPICS=$(kafka-topics --zookeeper [ZK_IP]:2181/kafka --list ). for T in $TOPICS. do. Kafka Delete Topic – Every message Apache Kafka receives stores it in log and by default, it keeps the messages for 168 hrs which is 7 days. To delete the topic or all its messages can be done in several ways and the rest of the article explains these.

Please add single quote ('giorgos-.* ') if it complains like no matches found: giorgos-.*

./bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic 'giorgos-.*'

Topic Deletion · The Internals of Apache Kafka, In the last few versions of Apache's Kafka, deleting a topic is fairly easy. You just need to set one property in the configuration to 'true', and just  Topic Deletion is a feature of Kafka that allows for deleting topics. +. TopicDeletionManager is responsible for topic deletion. +. Topic deletion is controlled by delete.topic.enable Kafka property that turns it on when true. +. Start a Kafka broker with broker ID 100. +.

Script to delete all kafka topics on a given cluster · GitHub, I created a Kafka topic and pushed large messages to that kafka topic. Now I am getting Jun 4, 2019 in Apache Kafka by sarvesh • 254 views. Whatever the reason, today I’ll show you a few ways to delete some or all the records from a Kafka topic. It should go without saying that you should use your best judgment and check (at least) twice before using the methods described below in a production environment. Kafka-delete-records. This command is available as part of Kafka CLI tools.

Manually delete Apache Kafka topics - Sunny Srinidhi, Kafka Delete Topic - Every message Apache Kafka receives stores it in log and by default, it keeps the messages for 168 hrs which is 7 days. In simple terms, Apache Kafka will keep latest version of a record and delete the older versions with same key. Kafka log compaction allows consumers to regain their state from compacted topic.

Kafka Delete Topic and its messages, Apache Kafka Topic and its Partition. As we know Kafka replicates each message multiple times on different servers for fault tolerance. So, even if one of the  Kafka can serve as a kind of external commit-log for a distributed system. The log helps replicate data between nodes and acts as a re-syncing mechanism for failed nodes to restore their data. The log compaction feature in Kafka helps support this usage. In this usage Kafka is similar to Apache BookKeeper project. 1.3 Quick Start

Comments
  • how about deleing multiple patterns, e.g., foo.* bar.* ?
  • Regex is always possible. For this example, giorgos-topic[123]. You can use OR too. (some-topic|my-other-topic_1)