hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads
Date Mon, 17 Dec 2012 02:36:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13533606#comment-13533606
] 

Lars Hofhansl commented on HBASE-7336:
--------------------------------------

The latest test run passed.

Looking at testConcurrentReading, it is a time bounded test. Checking the number of blocks
that the concurrent readers manage to read within the time bound is actually *larger* without
this patch (i.e. this patch is slowing this down)... Presumably because more threads are now
using pread.
(This test is reading random blocks randomly choosing pread vs not from many threads. And
these blocks are very small too. So not sure how real world that is.)

On the other hand, without this patch scanners that scan large HFiles concurrently in a tight
loop are useless (i.e. never make enough progress to not time out).

                
> HFileBlock.readAtOffset does not work well with multiple threads
> ----------------------------------------------------------------
>
>                 Key: HBASE-7336
>                 URL: https://issues.apache.org/jira/browse/HBASE-7336
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Lars Hofhansl
>            Assignee: Lars Hofhansl
>            Priority: Critical
>             Fix For: 0.96.0, 0.94.4
>
>         Attachments: 7336-0.94.txt, 7336-0.96.txt
>
>
> HBase grinds to a halt when many threads scan along the same set of blocks and neither
read short circuit is nor block caching is enabled for the dfs client ... disabling the block
cache makes sense on very large scans.
> It turns out that synchronizing in istream in HFileBlock.readAtOffset is the culprit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message