kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jun Rao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-405) Improve the high water mark maintenance to store high watermarks for all partitions in a single file on disk
Date Wed, 01 Aug 2012 18:11:03 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426790#comment-13426790
] 

Jun Rao commented on KAFKA-405:
-------------------------------

Just had a look of v4. A couple of minor comments:

40. HighwaterMarkCheckpoint:
40.1 If tempHwFile already exists, we can just overwrite it since we know hwFile is always
safe.
40.2 There is no need to delete hwFile first and then rename tempHwFile to it. Rename should
do the deletion. Currently, if we fail at the bad time, we could end up without a hwFile.
                
> Improve the high water mark maintenance to store high watermarks for all partitions in
a single file on disk
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-405
>                 URL: https://issues.apache.org/jira/browse/KAFKA-405
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.8
>            Reporter: Neha Narkhede
>            Assignee: Neha Narkhede
>         Attachments: kafka-405-v1.patch, kafka-405-v2.patch, kafka-405-v3.patch, kafka-405-v4.patch
>
>
> KAFKA-46 introduced per partition leader high watermarks. But it stores those in one
file per partition. A more performant solution would be to store all high watermarks in a
single file on disk

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message