hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ryan rawson (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2856) TestAcidGuarantee broken on trunk
Date Thu, 13 Jan 2011 06:03:47 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12981140#action_12981140

ryan rawson commented on HBASE-2856:

Unfortunately the paranoia re: performance is borne out by direct
experience.  It will be an issue, and it will be a blocker and we
should deal with it right now. Since fixing it might require
architectural level changes in how we manage these things internally.
including up to and including using a different ID stream for atomic

Backing up a bit, the basic issue is that a handler thread cannot
complete and return to the client until the row-transaction it was
working on is visible to other clients. To do otherwise risks data
loss for ICV and inconsistent read-your-own-write scenarios for
clients.  But while waiting we are tying up a handler thread, and have
to wait on the longest pole HLog append (which can take seconds at
their worst!).  You end up with a RS level stall which is pretty ugly.

I dont want 2 sets of sequence numbers, but I am concerned that we
might need it.  Perhaps we can find a more elegant mechanism of
cheaply keeping track of which seqids are 'committed' and visible and
which are not.  Right now we use a simple 'read point' which acts like
a line in the sand.  Previous proposals called for a bitmask of the
last N numbers.  The problem with this is that deferred flushing
combined with non-deferred flushing would cause major problems, as the
last N we need to keep track of keeps on expanding.

Perhaps a reverse bitmask where we keep track of the PREVIOUS N tx
that are NOT committed might make more sense.  Implementing it
efficiently is another question.

> TestAcidGuarantee broken on trunk 
> ----------------------------------
>                 Key: HBASE-2856
>                 URL: https://issues.apache.org/jira/browse/HBASE-2856
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.89.20100621
>            Reporter: ryan rawson
>            Assignee: stack
>            Priority: Blocker
>             Fix For: 0.92.0
>         Attachments: 2856-v2.txt, 2856-v3.txt, acid.txt
> TestAcidGuarantee has a test whereby it attempts to read a number of columns from a row,
and every so often the first column of N is different, when it should be the same.  This is
a bug deep inside the scanner whereby the first peek() of a row is done at time T then the
rest of the read is done at T+1 after a flush, thus the memstoreTS data is lost, and previously
'uncommitted' data becomes committed and flushed to disk.
> One possible solution is to introduce the memstoreTS (or similarly equivalent value)
to the HFile thus allowing us to preserve read consistency past flushes.  Another solution
involves fixing the scanners so that peek() is not destructive (and thus might return different
things at different times alas).

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message