hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-2856) TestAcidGuarantee broken on trunk
Date Mon, 06 Dec 2010 23:26:11 GMT

     [ https://issues.apache.org/jira/browse/HBASE-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

stack updated HBASE-2856:

    Assignee: stack  (was: ryan rawson)

Ryan, Kannan, and myself chatted on this issue a while:

+ We'll add a new insertion sequence id into the KeyValue Key.  It will be inserted after
the current Timestamp and before KeyValue Type.  This new insertion sequence id will be used
resolving 'order' when all else matches in two KeyValue keys.
+ We'd use the last two bits of the KeyValue Type for 'version'.  We'll set bit 128 to denote
KeyValue version '2' (what we currently have is KeyValue version '1').
+ We'd adjust our comparators so they ignored these upper bits in Type so we can compare Type
1 and Type 2 KeyValues.
+ A total cluster restart will be needed.  We'll up the RPC version to be sure.
+ Post-restart, writes will be version 2 KeyValues.
+ Reads may be a mix of version 2 and version 1 KeyValues while regions are made of a mix
of old and new style HFiles.  This should be fine as long as comparators work properly witha
mix of version 1 and version 2 KVs.
+ We're thinking the new insertion sequence id will be fixed-length long.  A vint/vlong will
make for an awkward parse.

> TestAcidGuarantee broken on trunk 
> ----------------------------------
>                 Key: HBASE-2856
>                 URL: https://issues.apache.org/jira/browse/HBASE-2856
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.89.20100621
>            Reporter: ryan rawson
>            Assignee: stack
>            Priority: Blocker
>             Fix For: 0.92.0
> TestAcidGuarantee has a test whereby it attempts to read a number of columns from a row,
and every so often the first column of N is different, when it should be the same.  This is
a bug deep inside the scanner whereby the first peek() of a row is done at time T then the
rest of the read is done at T+1 after a flush, thus the memstoreTS data is lost, and previously
'uncommitted' data becomes committed and flushed to disk.
> One possible solution is to introduce the memstoreTS (or similarly equivalent value)
to the HFile thus allowing us to preserve read consistency past flushes.  Another solution
involves fixing the scanners so that peek() is not destructive (and thus might return different
things at different times alas).

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message