hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2957) Release row lock when waiting for wal-sync
Date Mon, 06 Sep 2010 18:19:34 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12906558#action_12906558

stack commented on HBASE-2957:

@Dhruba How does memstore.inflight get flipped to be memstore.committed?  Safely?  There are
some existing mechanisms that might be of help.  There is a read/write consistency class in
hbase http://hbase.apache.org/docs/r0.89.20100726/xref/org/apache/hadoop/hbase/regionserver/ReadWriteConsistencyControl.html#1
that is used around memstore updates to help keep the reads and scans 'clean'; i.e. prevents
reading of partial updates on a row (There are some known caveats).  And then in KeyValue,
there is a 'special' extra timestamp used ensuring memstore consistency: http://hbase.apache.org/docs/r0.89.20100726/xref/org/apache/hadoop/hbase/KeyValue.html#216

Would it be simpler adding an ICV batcher/catcher/reservoir/overspill that sat in front of
an actual ICV call?   It would accumulate ICVs while WAL was busy syncing.  The ICV resevoir
would be an optional facility; you'd ask for it by setting a flag on the ICV call.  On return,
the actual ICV would check the reservoir to see if any ICVs had been accumulating while it
was off syncing.  If any, it'd suck them all up and apply the ICVs in a batch.  We'd need
to add a new batch ICV call, one that added a block of ICVs to the WAL  and that then did
the memstore updates playing the content of the batch one at a time.  We'd return to the client
as soon as we'd added an ICV to the revervoir not waiting on WAL

> Release row lock when waiting for wal-sync
> ------------------------------------------
>                 Key: HBASE-2957
>                 URL: https://issues.apache.org/jira/browse/HBASE-2957
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver, wal
>    Affects Versions: 0.20.0
>            Reporter: Prakash Khemani
> Is there a reason to hold on to the row-lock while waiting for the WAL-sync to be completed
by the logSyncer thread?
> I think data consistency will be guaranteed even if the following happens (a) the row
lock is held while the row is updated in memory (b) the row lock is released after queuing
the KV record for WAL-syncing (c) the log-sync system guarantees that the log records for
any given row are synced in order (d) the HBase client only receives a success notification
after the sync completes (no change from the current state)
> I think this should be a huge win. For my use case, and I am sure for others,  the handler
thread spends the bulk of its row-lock critical section  time waiting for sync to complete.
> Even if the log-sync system cannot guarantee the orderly completion of sync records,
the "Don't hold row lock while waiting for sync" option should be available to HBase clients
on a per request basis.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message