hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Himanshu Vashishtha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8741) Mutations on Regions in recovery mode might have same sequenceIDs
Date Thu, 20 Jun 2013 01:37:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688757#comment-13688757
] 

Himanshu Vashishtha commented on HBASE-8741:
--------------------------------------------

Hmmm, this idea does make sense to me. 

So, a wal file will have sequenceId from bunch of regions, and its sequenceId will not be
in ascending order. But, that shouldn't be any problem. In case of multi wals, it should also
work.

[~sershe]: Why do you think it will complicate log cleanup? It seems that the current replay/splitting
should be able to digest it too, with minimal change. The only change I could see is passing
some context info about maximum possible number of entries a region could have, while opening
it to a new region server. We could use Enis way of having maximum number of entries per wal
(as Stack suggested), or a hint based on the size of wal file.

                
> Mutations on Regions in recovery mode might have same sequenceIDs
> -----------------------------------------------------------------
>
>                 Key: HBASE-8741
>                 URL: https://issues.apache.org/jira/browse/HBASE-8741
>             Project: HBase
>          Issue Type: Bug
>          Components: MTTR
>    Affects Versions: 0.95.1
>            Reporter: Himanshu Vashishtha
>            Assignee: Himanshu Vashishtha
>
> Currently, when opening a region, we find the maximum sequence ID from all its HFiles
and then set the LogSequenceId of the log (in case the later is at a small value). This works
good in recovered.edits case as we are not writing to the region until we have replayed all
of its previous edits. 
> With distributed log replay, if we want to enable writes while a region is under recovery,
we need to make sure that the logSequenceId > maximum logSequenceId of the old regionserver.
Otherwise, we might have a situation where new edits have same (or smaller) sequenceIds. 
> We can store region level information in the WALTrailer, than this scenario could be
avoided by:
> a) reading the trailer of the "last completed" file, i.e., last wal file which has a
trailer and,
> b) completely reading the last wal file (this file would not have the trailer, so it
needs to be read completely).
> In future, if we switch to multi wal file, we could read the trailer for all completed
WAL files, and reading the remaining incomplete files.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message