hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Himanshu Vashishtha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8741) Mutations on Regions in recovery mode might have same sequenceIDs
Date Sun, 23 Jun 2013 21:56:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13691620#comment-13691620

Himanshu Vashishtha commented on HBASE-8741:

So, the basic scheme looks like:
A region carries a sequenceID counter which will be used while inserting entries in the WAL.
On opening a region, it will be set to whatever the max flush sequenceId of the region is
(obtained from the hfiles). A WAL will have entries from various regions (so it is not in
increasing order now; that should be ok). When we roll the wal, we keep a data structure of
<WAL_file_path> : (<Region>:<Current_value_of_RegionSequenceId>>).

While archiving WALs, we need to make sure that we archive wals if all its entries are already
flushed. We would compare the above data structure (basically a map of region to sequenceId
at time of rolling) to the max(completeSequenceId, openSequenceId) for each region. If it
is less, then we no longer need that wal, and archive it; else we submit a flush request for
those regions whose entries are greater.

For doing writes doing failover, we need to give a hint to the new regionserver some value
(could be equal to current number of wals * max number of entries a wal could take). This
should be used to bump up the value of sequenceId counter of the region.
> Mutations on Regions in recovery mode might have same sequenceIDs
> -----------------------------------------------------------------
>                 Key: HBASE-8741
>                 URL: https://issues.apache.org/jira/browse/HBASE-8741
>             Project: HBase
>          Issue Type: Bug
>          Components: MTTR
>    Affects Versions: 0.95.1
>            Reporter: Himanshu Vashishtha
>            Assignee: Himanshu Vashishtha
> Currently, when opening a region, we find the maximum sequence ID from all its HFiles
and then set the LogSequenceId of the log (in case the later is at a small value). This works
good in recovered.edits case as we are not writing to the region until we have replayed all
of its previous edits. 
> With distributed log replay, if we want to enable writes while a region is under recovery,
we need to make sure that the logSequenceId > maximum logSequenceId of the old regionserver.
Otherwise, we might have a situation where new edits have same (or smaller) sequenceIds. 
> We can store region level information in the WALTrailer, than this scenario could be
avoided by:
> a) reading the trailer of the "last completed" file, i.e., last wal file which has a
trailer and,
> b) completely reading the last wal file (this file would not have the trailer, so it
needs to be read completely).
> In future, if we switch to multi wal file, we could read the trailer for all completed
WAL files, and reading the remaining incomplete files.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message