使用zookeeper-shall.sh rmr brokers /主題刪除主題和刪除Kafka10上kafka-topics.sh的標誌之間的區別

Question

從我在網上找到的和其他堆棧溢出帖子看來，主要有兩種刪除主題的方式在卡夫卡。 的第一點是：a）delete.topic.enable = true和運行./kafka-topics.sh ---delete --topic <topicName> 方式二：./zookeeper-shell.sh localhost:2181 rmr brokers/topics

我也注意到，第一種方法標記每個話題被刪除，在一兩分鐘的主題被刪除的地方作爲第二個方法刪除它們瞬間。我還注意到，重新啓動服務器需要幾個小時，這是正常的嗎？我在一家經紀商有超過1000個主題（用於測試目的）。

Answer 1

第一種方法將創建zookeper admin/delete_topics/<topic>一個節點，如果啓用了話題缺失像你一樣，在卡夫卡的經紀人給定線程（TopicDeletionManager），其監測delete_topics孩子的，將解決這個問題，這意味着從刪除動物園管理員，但也從所有卡夫卡副本日誌，確保你不會在一個無效的狀態。整個過程描述如下： https://github.com/apache/kafka/blob/0.11.0/core/src/main/scala/kafka/controller/TopicDeletionManager.scala

/** 
* This manages the state machine for topic deletion. 
* 1. TopicCommand issues topic deletion by creating a new admin path /admin/delete_topics/<topic> 
* 2. The controller listens for child changes on /admin/delete_topic and starts topic deletion for the respective topics 
* 3. The controller's ControllerEventThread handles topic deletion. A topic will be ineligible 
* for deletion in the following scenarios - 
    * 3.1 broker hosting one of the replicas for that topic goes down 
    * 3.2 partition reassignment for partitions of that topic is in progress 
* 4. Topic deletion is resumed when - 
* 4.1 broker hosting one of the replicas for that topic is started 
* 4.2 partition reassignment for partitions of that topic completes 
* 5. Every replica for a topic being deleted is in either of the 3 states - 
* 5.1 TopicDeletionStarted Replica enters TopicDeletionStarted phase when onPartitionDeletion is invoked. 
*  This happens when the child change watch for /admin/delete_topics fires on the controller. As part of this state 
*  change, the controller sends StopReplicaRequests to all replicas. It registers a callback for the 
*  StopReplicaResponse when deletePartition=true thereby invoking a callback when a response for delete replica 
*  is received from every replica) 
* 5.2 TopicDeletionSuccessful moves replicas from 
*  TopicDeletionStarted->TopicDeletionSuccessful depending on the error codes in StopReplicaResponse 
* 5.3 TopicDeletionFailed moves replicas from 
*  TopicDeletionStarted->TopicDeletionFailed depending on the error codes in StopReplicaResponse. 
*  In general, if a broker dies and if it hosted replicas for topics being deleted, the controller marks the 
*  respective replicas in TopicDeletionFailed state in the onBrokerFailure callback. The reason is that if a 
*  broker fails before the request is sent and after the replica is in TopicDeletionStarted state, 
*  it is possible that the replica will mistakenly remain in TopicDeletionStarted state and topic deletion 
*  will not be retried when the broker comes back up. 
* 6. A topic is marked successfully deleted only if all replicas are in TopicDeletionSuccessful 
* state. Topic deletion teardown mode deletes all topic state from the controllerContext 
* as well as from zookeeper. This is the only time the /brokers/topics/<topic> path gets deleted. On the other hand, 
* if no replica is in TopicDeletionStarted state and at least one replica is in TopicDeletionFailed state, then 
* it marks the topic for deletion retry. 

直接從動物園管理員刪除只是意味着從協調器中刪除。當請求元數據時，主題不在這裏了（好吧，也許它們可能來自緩存），但日誌文件不會被刪除（至少現在不是，我假設經紀商會檢測到日誌無效並在某個時間刪除它們），但是你可能會對經紀人有一些不協調的地方（如果你在重新平衡中，你可能會破壞很多東西）。這可能意味着一些經紀人會認爲它被刪除，而另一些經紀人會認爲它仍然存在......遠非理想。

從現在開始刪除fom zookeeper（和來自中間商的日誌）似乎確實有可能，但要小心，它可能會造成衝突，無效狀態，隨機錯誤，並且在將來的版本中它可能根本不起作用。