hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-3504) HLog performance improvement
Date Fri, 04 Feb 2011 20:13:30 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12990720#comment-12990720

Jonathan Gray commented on HBASE-3504:

There is still some extra stuff inside the updateLock like a number of currentTimeMillis and
other metrics-only operations.  These can be pulled out.  Unfortunately some of it would require
a bit of refactoring.

The work done inside the log-rolling is not really the issue, it's that every single write
operation on the server requires grabbing a single, exclusive lock.  That's going to be (and
is) a bottleneck.  We've already done work around removing other locking bottlenecks, for
example, around onlineRegions and the row lock data structures.  It showed marked improvements
(when running w/o WAL).  Patches on all that forthcoming.

A much bigger improvement will come from removing the exclusive lock and changing it to a
read/write lock.  I've gone through the code multiple times and I'm not convinced that having
out-of-order seqids is a problem as long as we retain the properties I described above.

> HLog performance improvement
> ----------------------------
>                 Key: HBASE-3504
>                 URL: https://issues.apache.org/jira/browse/HBASE-3504
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
> The HLog.updateLock protects the rolling of logs with concurrent writes to the HDFS log
file. This is a scalability bottleneck for a workload that comprises mostly of counter-increments.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message