kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jun Rao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-1074) Reassign partitions should delete the old replicas from disk
Date Wed, 04 Dec 2013 18:16:36 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-1074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13839154#comment-13839154
] 

Jun Rao commented on KAFKA-1074:
--------------------------------

Summary of the new patch:
LogCleaner:
*added the logic to cancel an in-progress cleaning task
*made the cleaner thread a shutdownable, but not interruptible thread. Interrupting the cleaner
may cause the file channel to be closed, which will fail other operations like log flushing
during shutdown.

LogManager:
* added the logic to wait until an in-progress flushing process complete
* to delete a log do
  (1) take the log to be deleted from log lists so that log cleaner and log flusher won't
see it in the future
  (2) cancel any in-progress cleaning task
  (3) wait until any in-progress flushing process to complete
  (4) at this moment, we know the log won't be touched by the cleaner or the flusher any more
and we can delete the whole directory synchronously.

Todos:
* If the overall logic looks good, we can get rid of OptimisticLockFailureException in LogCleaner
too by canceling any cleaner task before truncating the log. This can be done in a follow
up jira.

> Reassign partitions should delete the old replicas from disk
> ------------------------------------------------------------
>
>                 Key: KAFKA-1074
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1074
>             Project: Kafka
>          Issue Type: Bug
>          Components: core
>    Affects Versions: 0.8.0
>            Reporter: Jun Rao
>            Assignee: Jun Rao
>             Fix For: 0.8.1
>
>         Attachments: KAFKA-1074.patch, KAFKA-1074_2013-12-04_10:14:13.patch
>
>
> Currently, after reassigning replicas to other brokers, the old replicas are not removed
from disk and have to be deleted manually.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message