hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6758) [replication] The replication-executor should make sure the file that it is replicating is closed before declaring success on that file
Date Wed, 10 Oct 2012 21:51:03 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473593#comment-13473593
] 

Devaraj Das commented on HBASE-6758:
------------------------------------

[~jdcryans], this sequence of events could happen currently too, isn't it? The lines of code
that I moved are to do with postLogRoll which happens after the sequence that you are talking
about. This problem exists with/without this patch.

bq. You end up with a log tracked in ZK that doesn't exist. This RS's queue will be recovered
by another RS that will eventually try to read from that non-existing file. My concern is
how we're going to treat that file.

To answer your question, I think the RS that picks this queue up will dump the file after
a couple of retries (since the file doesn't exist and will never show up in the recovered
logs directory).
                
> [replication] The replication-executor should make sure the file that it is replicating
is closed before declaring success on that file
> ---------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-6758
>                 URL: https://issues.apache.org/jira/browse/HBASE-6758
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Devaraj Das
>            Assignee: Devaraj Das
>            Priority: Critical
>             Fix For: 0.96.0
>
>         Attachments: 6758-1-0.92.patch, 6758-2-0.92.patch, 6758-trunk-1.patch, 6758-trunk-2.patch,
6758-trunk-3.patch, 6758-trunk-4.patch, TEST-org.apache.hadoop.hbase.replication.TestReplication.xml
>
>
> I have seen cases where the replication-executor would lose data to replicate since the
file hasn't been closed yet. Upon closing, the new data becomes visible. Before that happens
the ZK node shouldn't be deleted in ReplicationSourceManager.logPositionAndCleanOldLogs. Changes
need to be made in ReplicationSource.processEndOfFile as well (currentPath related).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message