hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4487) The increment operation can release the rowlock before sync-ing the Hlog
Date Fri, 30 Sep 2011 05:26:45 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13117889#comment-13117889

dhruba borthakur commented on HBASE-4487:

> This is like a group commit in HLog. We have a group committing going on inside in sync
down in dfsclient.

The DFSClient write code path is pretty heavyweight (as it stands now), as shown in the code
DFSClient.writeChunk(). It acquires a synchronized section, creates packets, stuffs crcs in
it, acquires locks for the outgoing queue and queues the packet. And all this is done while
holding the  HLog.updateLock, which means it blocks out any other writes to the HLog for this
time. Instead, this patch queues the write-data completely inside the hbase code and the HLog.updateLock
is released much much earlier.

> If I read the patch correctly it adds new sync calls that are performed with the updateLock

There are no additional code paths that this patch introduced where a HLog.sync occurs within
the HLog.updateLock. The only pre-existing place where a sync calls occurs within the HLog.updateLock
is in HLog.cleanupCurrentWriter. This code path is invoked only during a logroll; and during
a logroll all currently executing transactions are anyways quiesced, so I see no impact to
concurrency in this code path. Please let me know if you disagree.

> if (this.closed) : Is this check not needed here then?

I think this code is still needed, this check was not introduced by this patch. Any reasons
why you think it might not be needed?

> Can we pass new config parameter through conf which enables this new feature ?

I can surely make this configurable. But I am doubtful if anybody would care to run with this
feature switched off (since it is a performance fix with potentially no downsides for any
usecase). But if you feel strongly about it, I will make it configurable. However, most of
the common code in HLog will not be inside this configurable parameter, only the change in
HRegion.increment() can be switched on/off via this new config. Please let me know.

> The increment operation can release the rowlock before sync-ing the Hlog
> ------------------------------------------------------------------------
>                 Key: HBASE-4487
>                 URL: https://issues.apache.org/jira/browse/HBASE-4487
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>         Attachments: appendNoSync4.txt, appendNoSync5.txt, appendNoSync6.txt
> This allows for better throughput when there are hot rows.I have seen this change make
a single row update improve from 400 increments/sec/server to 4000 increments/sec/server.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message