hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads
Date Wed, 12 Dec 2012 06:01:24 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529647#comment-13529647

Lars Hofhansl commented on HBASE-7336:

I have a patch, that changes that synchronized to a ReentrantLock and reverts to pread if
the lock cannot be acquired.
That way we get seek + read when possible and pread when necessary or requested. Not that
this is only for scans. Get default to pread anyway.

With that patch I get the following behavior:
One client: 15s
Two clients: 22s each

Which seems reasonable given above numbers.

I wouldn't really consider this a proper fix, although I also wouldn't know what the proper
fix should be.
The limiting factor here is a single FSInputStream per store file, which can be devastating
for performance if many threads read the same store file and it does not fit into the block
cache (or the scans chose not to cache the blocks).

> HFileBlock.readAtOffset does not work well with multiple threads
> ----------------------------------------------------------------
>                 Key: HBASE-7336
>                 URL: https://issues.apache.org/jira/browse/HBASE-7336
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Lars Hofhansl
>            Assignee: Lars Hofhansl
>            Priority: Critical
>             Fix For: 0.96.0, 0.94.4
> HBase grinds to a halt when many threads scan along the same set of blocks and neither
read short circuit is nor block caching is enabled for the dfs client ... disabling the block
cache makes sense on very large scans.
> It turns out that synchronizing in istream in HFileBlock.readAtOffset is the culprit.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message