hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1213) Investigate Locking & Contention in the Write Path
Date Thu, 07 May 2009 20:40:45 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707088#action_12707088

stack commented on HBASE-1213:

This is about carrying batches deeper down into the Region.  We'd then get lock on HLog, run
a batch, let the HLog lock go and then apply the commit logged edits across Stores.

This is going to become even more critical as HLog becomes more expensive when flush/sync
actually starts working.

Currently, edits go in a row at a time.  The commit log and the commit to memcache are both
done within the row lock.  We'd break this batching HLog writes.  Would make it so commit
log had edits not yet added to memcache.  This is probably OK.

Of note, if a row has many edits to apply -- the BatchUpdate touches many columns -- these
are applied as a batch to HLog and then to the memcache.

If we could presume the array of incoming edits already sorted by region, then we could pass
sub-arrays down to Regions -- no need to sort first or repackage in a new array or List.

We'd need a new batchUpdate method in HRegion, one that took sub-arrays of Region edits. 
It'd pass the edits to the HLog first, then one by one, call current batchUpdate only with
writeToWAL set to false.

> Investigate Locking & Contention in the Write Path
> --------------------------------------------------
>                 Key: HBASE-1213
>                 URL: https://issues.apache.org/jira/browse/HBASE-1213
>             Project: Hadoop HBase
>          Issue Type: Improvement
>    Affects Versions: 0.19.0
>            Reporter: Ben Maurer
>            Assignee: stack
>             Fix For: 0.20.0
> When doing a large number of bulk updates from different clients, I noticed that there
was a high level of lock contention for stuff like locking the HLog. It seems that each thread
acquires the lock for a single BatchUpdate, releases the lock then another thread owns the
lock before the initial writer gets to the next update. Having the threads bounce around may
lead to suboptimal performance.
> Should be benchmarked & maybe changed to have less context switching.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message