hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ryan rawson (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-2957) Release row lock when waiting for wal-sync
Date Wed, 08 Sep 2010 18:43:34 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907357#action_12907357
] 

ryan rawson commented on HBASE-2957:
------------------------------------

Prakash, I think we must be talking about different things, because delayed syncs _do_ guarantee
consistency. 

Some background on the code... HLog is a sequence of entries... while concurrent writes have
indeterminate ordering, writes on the same row lock are effective serialized and thus there
is a strong causal ordering between HLog entries and sequence of operations for writes under
the same row lock.  If you look at the implementation of HLog, there is a sync block in the
middle in append which causes all entries to have a serial sequence id and also protects the
underlying file which is not multithread capable.

The way that delayed syncs work is like so for ICVs:

Begin Row Lock
Do Get
Make new Value from Old Value and increment Amount
Append to HLog
if (!delayed sync) Do HLog sync
Update Memstore
Release Row Lock

The longest pole item in this sequence tends to be the hlog sync.  So make it optional.  The
entries are now buffered in memory.  The next sync by either a client on a different table,
or via the background thread timer will flush the entries to HDFS thus persisting* them.

The ordering in the hlog will be the same with or without delayed sync, and performance is
vastly improved at a small hit in durability.  For peacemeal ICVs, losing 100ms now and again
should not be statistically significant in high volume cases.  The setting is on a per-table
basis, thus you can choose your tradeoff at that level.

> Release row lock when waiting for wal-sync
> ------------------------------------------
>
>                 Key: HBASE-2957
>                 URL: https://issues.apache.org/jira/browse/HBASE-2957
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver, wal
>    Affects Versions: 0.20.0
>            Reporter: Prakash Khemani
>
> Is there a reason to hold on to the row-lock while waiting for the WAL-sync to be completed
by the logSyncer thread?
> I think data consistency will be guaranteed even if the following happens (a) the row
lock is held while the row is updated in memory (b) the row lock is released after queuing
the KV record for WAL-syncing (c) the log-sync system guarantees that the log records for
any given row are synced in order (d) the HBase client only receives a success notification
after the sync completes (no change from the current state)
> I think this should be a huge win. For my use case, and I am sure for others,  the handler
thread spends the bulk of its row-lock critical section  time waiting for sync to complete.
> Even if the log-sync system cannot guarantee the orderly completion of sync records,
the "Don't hold row lock while waiting for sync" option should be available to HBase clients
on a per request basis.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message