hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Rodionov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads
Date Tue, 15 Jul 2014 17:23:07 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14062356#comment-14062356

Vladimir Rodionov commented on HBASE-7336:

h3. Effect of compaction on large scan operations and vice verse

The compaction scanner and user scanner will compete for the same input stream (DFSInputStream).
This results in a sub optimal performance for both of them, becuase *there is no guarantee
that next call to read HFile block from the "lucky" scanner will use the same streaming API
and pre-cached data will still be valid*. Yep? Both scanners, periodically, switch between
stream/pread API calls, hdfs cache can not be used (?), performance of both of them is defined
by positional read performance (which is low for scan mode operation).

Is this correct assessment?    

> HFileBlock.readAtOffset does not work well with multiple threads
> ----------------------------------------------------------------
>                 Key: HBASE-7336
>                 URL: https://issues.apache.org/jira/browse/HBASE-7336
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Performance
>            Reporter: Lars Hofhansl
>            Assignee: Lars Hofhansl
>            Priority: Critical
>             Fix For: 0.94.4, 0.95.0
>         Attachments: 7336-0.94.txt, 7336-0.96.txt
> HBase grinds to a halt when many threads scan along the same set of blocks and neither
read short circuit is nor block caching is enabled for the dfs client ... disabling the block
cache makes sense on very large scans.
> It turns out that synchronizing in istream in HFileBlock.readAtOffset is the culprit.

This message was sent by Atlassian JIRA

View raw message