hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Feng Honghua (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-9465) HLog entries are not pushed to peer clusters serially when region-move or RS failure in master cluster
Date Tue, 10 Sep 2013 02:51:52 GMT

    [ https://issues.apache.org/jira/browse/HBASE-9465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13762665#comment-13762665
] 

Feng Honghua commented on HBASE-9465:
-------------------------------------

[~lhofhansl]

For RS failure scenario, can we delay the assigning of recovered regions until all the remained
hlog files of the failed RS are pushed to peer clusters (the hlog split can be parallel with
the hlog push though)? This way we can maintain the (global) serial push for hlog entries
of a region even in face of RS failure.

But for region-move it's harder to maintain global serial push since it's harder to determine
all the hlog entries of a given region has been pushed to peer clusters when the containing
RS is healthy and continuously receiving write requests.
                
> HLog entries are not pushed to peer clusters serially when region-move or RS failure
in master cluster
> ------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-9465
>                 URL: https://issues.apache.org/jira/browse/HBASE-9465
>             Project: HBase
>          Issue Type: Bug
>          Components: regionserver, Replication
>            Reporter: Feng Honghua
>
> When region-move or RS failure occurs in master cluster, the hlog entries that are not
pushed before region-move or RS-failure will be pushed by original RS(for region move) or
another RS which takes over the remained hlog of dead RS(for RS failure), and the new entries
for the same region(s) will be pushed by the RS which now serves the region(s), but they push
the hlog entries of a same region concurrently without coordination.
> This treatment can possibly lead to data inconsistency between master and peer clusters:
> 1. there are put and then delete written to master cluster
> 2. due to region-move / RS-failure, they are pushed by different replication-source threads
to peer cluster
> 3. if delete is pushed to peer cluster before put, and flush and major-compact occurs
in peer cluster before put is pushed to peer cluster, the delete is collected and the put
remains in peer cluster
> In this scenario, the put remains in peer cluster, but in master cluster the put is masked
by the delete, hence data inconsistency between master and peer clusters

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message