hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads
Date Wed, 02 Jul 2014 22:13:25 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14050791#comment-14050791

Lars Hofhansl commented on HBASE-7336:

I agree with your assessment on the seek+read vs pread. The current should not be worse than
doing all pread, though.

I see you commented on HBASE-5979 as well, and that would be a correct way to fix this. Each
scanner would have its own stream and hence seek+read should be better there.
The issue there is invalidation after a compaction or flush, although that needs some fixing
anyway - I tried (unsuccessfully) in HBASE-10060. The issue there is that even the memory
barriers taken for a lock that is uncontended 99.9999% of the time is a significant performance
problem, but I have not been able to remove it and still ensure correct behavior.

I'm glad you're looking at this, this area needs some TLC.

> HFileBlock.readAtOffset does not work well with multiple threads
> ----------------------------------------------------------------
>                 Key: HBASE-7336
>                 URL: https://issues.apache.org/jira/browse/HBASE-7336
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Performance
>            Reporter: Lars Hofhansl
>            Assignee: Lars Hofhansl
>            Priority: Critical
>             Fix For: 0.94.4, 0.95.0
>         Attachments: 7336-0.94.txt, 7336-0.96.txt
> HBase grinds to a halt when many threads scan along the same set of blocks and neither
read short circuit is nor block caching is enabled for the dfs client ... disabling the block
cache makes sense on very large scans.
> It turns out that synchronizing in istream in HFileBlock.readAtOffset is the culprit.

This message was sent by Atlassian JIRA

View raw message