hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6758) [replication] The replication-executor should make sure the file that it is replicating is closed before declaring success on that file
Date Wed, 10 Oct 2012 21:13:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473566#comment-13473566
] 

Devaraj Das commented on HBASE-6758:
------------------------------------

bq. Ah I see, I didn't fully grok the new preRoll/postRoll dance in my first review. That's
clever.

Cool. Thanks for taking a pass at this.

bq. Will the recovered queue hang or will it abandon that HLog? FWIW there's another jira
regarding that problem but this could be a new failure case.

The change done to the placement of the postLogRoll call in the patch will not affect recovered
queues. This will only affect files that the RS in question is creating himself. The changes
in ReplicationSource.java will only take effect for non-recovered files (there is a check
_!this.queueRecovered_ before setting _currentWALisBeingWrittenTo_ to true).. So I think we
are covered (please let me know if I missed something or misunderstood your concern).

I'll submit a patch shortly with the nits pointed out by [~tedyu@apache.org] fixed.
                
> [replication] The replication-executor should make sure the file that it is replicating
is closed before declaring success on that file
> ---------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-6758
>                 URL: https://issues.apache.org/jira/browse/HBASE-6758
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Devaraj Das
>            Assignee: Devaraj Das
>            Priority: Critical
>             Fix For: 0.96.0
>
>         Attachments: 6758-1-0.92.patch, 6758-2-0.92.patch, 6758-trunk-1.patch, 6758-trunk-2.patch,
6758-trunk-3.patch, TEST-org.apache.hadoop.hbase.replication.TestReplication.xml
>
>
> I have seen cases where the replication-executor would lose data to replicate since the
file hasn't been closed yet. Upon closing, the new data becomes visible. Before that happens
the ZK node shouldn't be deleted in ReplicationSourceManager.logPositionAndCleanOldLogs. Changes
need to be made in ReplicationSource.processEndOfFile as well (currentPath related).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message