hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jean-Daniel Cryans (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6758) [replication] The replication-executor should make sure the file that it is replicating is closed before declaring success on that file
Date Mon, 17 Sep 2012 21:21:08 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13457342#comment-13457342
] 

Jean-Daniel Cryans commented on HBASE-6758:
-------------------------------------------

I see, all that double-negation (eg !fileNotInUse) confused me :)

So in layman's terms, your patch short circuits all the checks to change the current path
if we know for sure that the file we are replicating from is being written to. The side effect
is that we won't quit the current file unless it has aged right? 

bq. The replication executor is always trailing, and so when the HLog guy says that a path
is not in use (being written to), it seems to me a fact that it indeed is not being written
to and any writes that ever happened was in the past.

FWIW that might not be totally true, at least in 0.94 HLog.postLogRoll is called before HLog.cleanupCurrentWriter
which does issue a sync().


                
> [replication] The replication-executor should make sure the file that it is replicating
is closed before declaring success on that file
> ---------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-6758
>                 URL: https://issues.apache.org/jira/browse/HBASE-6758
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Devaraj Das
>            Assignee: Devaraj Das
>         Attachments: 6758-1-0.92.patch
>
>
> I have seen cases where the replication-executor would lose data to replicate since the
file hasn't been closed yet. Upon closing, the new data becomes visible. Before that happens
the ZK node shouldn't be deleted in ReplicationSourceManager.logPositionAndCleanOldLogs. Changes
need to be made in ReplicationSource.processEndOfFile as well (currentPath related).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message