hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Duo Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17407) Correct update of maxFlushedSeqId in HRegion
Date Wed, 04 Jan 2017 13:33:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15798278#comment-15798278

Duo Zhang commented on HBASE-17407:

And I have claimed many times, you need to think about the future of the project. We should
not plant bombs in the code...

The lowestUnflushedSequenceIds, as its name, is used to store the lowest unflushed sequence
id of a store. In the old time, we will always flush all the contents of a memstore for a
store, so we will just remove the entries in it and when new edits come it will put its value
into it. And in general, we will abort the RS if flush fails as we can not put the snapshot
back because the memstore may already have some new data. The exception is when the RS is
assigned a new region which need to be recovered from a crash. Before the recovery done, we
will not accept any writes so it is OK to just put back the snapshot when flush fails. And
IMO, the flush for recovery should be much simpler so maybe we should not let it share code
with the normal flush. So in the normal path, lowestUnflushedSequenceIds will never go backward
and actually, it is not something transactional. After we remove the entry from lowestUnflushedSequenceIds
it will be filled by new edits soon. So if you do not flush all the contents in memstore for
a store, I think the right way is to get the correct lowest unflushed sequence id at the first
place and store it into lowestUnflushedSequenceIds. I do not know why you guys would like
to remove the value in it and let others store an incorrect value in it and at last override
it with the correct value. What is the benefits? The {{onlyIfGreater}} parameter makes me
really nervous...


> Correct update of maxFlushedSeqId in HRegion
> --------------------------------------------
>                 Key: HBASE-17407
>                 URL: https://issues.apache.org/jira/browse/HBASE-17407
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Eshcar Hillel
> The attribute maxFlushedSeqId in HRegion is used to track the max sequence id in the
store files and is reported to HMaster. When flushing only part of the memstore content this
value might be incorrect and may cause data loss.

This message was sent by Atlassian JIRA

View raw message