hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Duo Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13811) Splitting WALs, we are filtering out too many edits -> DATALOSS
Date Thu, 04 Jun 2015 00:46:38 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14571917#comment-14571917

Duo Zhang commented on HBASE-13811:

flushOpSeqId = getNextSequenceId(wal);
flushedSeqId = getFlushedSequenceId(encodedRegionName, flushOpSeqId);

I think the problem is here...
Before the patch, getFlushedSequenceId will return flushOpSeqId because getEarliestMemstoreSeqNum
will return HConstants.NO_SEQNUM. But now we modify getEarliestMemstoreSeqNum, it will also
consider the sequenceIds recorded in flushingSequenceIds so it will not return HConstants.NO_SEQNUM
even if we decided to flush all stores. This may cause we replay unnecessary edits I think.

So the problem is here we only want to consider the sequenceIds in lowestUnflushedSequenceIds,
so maybe a new method?

Thanks. [~stack]

> Splitting WALs, we are filtering out too many edits -> DATALOSS
> ---------------------------------------------------------------
>                 Key: HBASE-13811
>                 URL: https://issues.apache.org/jira/browse/HBASE-13811
>             Project: HBase
>          Issue Type: Bug
>          Components: wal
>    Affects Versions: 2.0.0, 1.2.0
>            Reporter: stack
>            Assignee: stack
>            Priority: Critical
>             Fix For: 2.0.0, 1.2.0
>         Attachments: 13811.branch-1.txt, 13811.branch-1.txt, 13811.txt, HBASE-13811-v1.testcase.patch,
> I've been running ITBLLs against branch-1 around HBASE-13616 (move of ServerShutdownHandler
to pv2). I have come across an instance of dataloss. My patch for HBASE-13616 was in place
so can only think it the cause (but cannot see how). When we split the logs, we are skipping
legit edits. Digging.

This message was sent by Atlassian JIRA

View raw message