hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ryan rawson (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2856) TestAcidGuarantee broken on trunk
Date Tue, 01 Feb 2011 21:48:29 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12989399#comment-12989399

ryan rawson commented on HBASE-2856:

Don't we already have this? The comparator uses the max_seq_id to break ties between KVs...

The primary issue is that we need to know which KVs are 'committed' and which are still being
created in progress. Right now we have a problem whereby the scanner stack gets a little wonky
about how it handles partial next()s. By moving the memstoreTS pruning up to the HRegion scanner
level, and working on entire rows at a time, this might mitigate most of the problem actually.
This might get ugly with family only flushes, since in theory you might end up with a row
that is not completely written but is in memstore & hfile at the same time. Given that
the scope of a RWCC "transaction" is only memstore insert, I'm not sure how that would happen.
It's possible we could prevent it from becoming a problem with judicious use of the updateLock
in HRegion though.

For example, by grabbing the updateLock.writeLock().lock() during the switch over, or the
flush, we could ensure that all the pending writes are now complete, then do the switch out,
then we'd never have a situation where a half committed write is in memstore & hfile at
the same time.

> TestAcidGuarantee broken on trunk 
> ----------------------------------
>                 Key: HBASE-2856
>                 URL: https://issues.apache.org/jira/browse/HBASE-2856
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.89.20100621
>            Reporter: ryan rawson
>            Assignee: stack
>            Priority: Blocker
>             Fix For: 0.92.0
>         Attachments: 2856-v2.txt, 2856-v3.txt, 2856-v4.txt, 2856-v5.txt, acid.txt
> TestAcidGuarantee has a test whereby it attempts to read a number of columns from a row,
and every so often the first column of N is different, when it should be the same.  This is
a bug deep inside the scanner whereby the first peek() of a row is done at time T then the
rest of the read is done at T+1 after a flush, thus the memstoreTS data is lost, and previously
'uncommitted' data becomes committed and flushed to disk.
> One possible solution is to introduce the memstoreTS (or similarly equivalent value)
to the HFile thus allowing us to preserve read consistency past flushes.  Another solution
involves fixing the scanners so that peek() is not destructive (and thus might return different
things at different times alas).

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message