hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Himanshu Vashishtha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8741) Scope sequenceid to the region rather than regionserver (WAS: Mutations on Regions in recovery mode might have same sequenceIDs)
Date Thu, 17 Oct 2013 19:12:45 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13798296#comment-13798296

Himanshu Vashishtha commented on HBASE-8741:

Right, the pre-commit build which failed above was using 32 bit vm. The original issue was
a missing boolean member in heap size report, and I filed HBASE-9785 for that. Its description
explains what was happening in the precommit build on 32 bit with this patch. Here is a patch
that includes that fix. Lets it get tested on the pre-commit build. Thanks.

Yes, this patch ensures that sequenceIds for a region are always appended to the WAL in an
increasing order. 
Also, TestHLog#testConcurrentWrites() runs a test where three threads are writing to one region
concurrently. HLogPerformanceEvaluation verify method ensures that the records inserted are
in order. 

Thanks a lot for taking a look.

> Scope sequenceid to the region rather than regionserver (WAS: Mutations on Regions in
recovery mode might have same sequenceIDs)
> --------------------------------------------------------------------------------------------------------------------------------
>                 Key: HBASE-8741
>                 URL: https://issues.apache.org/jira/browse/HBASE-8741
>             Project: HBase
>          Issue Type: Bug
>          Components: MTTR
>    Affects Versions: 0.95.1
>            Reporter: Himanshu Vashishtha
>            Assignee: Himanshu Vashishtha
>             Fix For: 0.98.0
>         Attachments: HBASE-8741-trunk-v6.patch, HBASE-8741-v0.patch, HBASE-8741-v2.patch,
HBASE-8741-v3.patch, HBASE-8741-v4-again.patch, HBASE-8741-v4-again.patch, HBASE-8741-v4.patch,
HBASE-8741-v5-again.patch, HBASE-8741-v5.patch
> Currently, when opening a region, we find the maximum sequence ID from all its HFiles
and then set the LogSequenceId of the log (in case the later is at a small value). This works
good in recovered.edits case as we are not writing to the region until we have replayed all
of its previous edits. 
> With distributed log replay, if we want to enable writes while a region is under recovery,
we need to make sure that the logSequenceId > maximum logSequenceId of the old regionserver.
Otherwise, we might have a situation where new edits have same (or smaller) sequenceIds. 
> We can store region level information in the WALTrailer, than this scenario could be
avoided by:
> a) reading the trailer of the "last completed" file, i.e., last wal file which has a
trailer and,
> b) completely reading the last wal file (this file would not have the trailer, so it
needs to be read completely).
> In future, if we switch to multi wal file, we could read the trailer for all completed
WAL files, and reading the remaining incomplete files.

This message was sent by Atlassian JIRA

View raw message