hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12411) Optionally enable p-reads and private readers for compactions
Date Wed, 19 Nov 2014 05:34:34 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14217447#comment-14217447
] 

Lars Hofhansl commented on HBASE-12411:
---------------------------------------

Some numbers: 
||scanners per region||pread forced||OS cache||time/s||
|1|yes|no|33|
|1|yes|yes|30|
|1|no|no|22|
|1|no|yes|20|
|5|no|yes|13|
|5|no|no|13.3|
|5|yes|yes|13.2|
|5|no|no|13.4|
|30|no|yes|12.9|
|30|yes|yes|12.8|
|30|yes|no|13.4|
|30|no|no|13.1|

Note: In the 1 scanner per region case I see no visible load on the data node process. In
all other cases, due to the involved p-reads I see considerable load on the DN.

So it looks like p-read everywhere is not of much help. 


> Optionally enable p-reads and private readers for compactions
> -------------------------------------------------------------
>
>                 Key: HBASE-12411
>                 URL: https://issues.apache.org/jira/browse/HBASE-12411
>             Project: HBase
>          Issue Type: Improvement
>          Components: Performance
>            Reporter: Lars Hofhansl
>            Assignee: Lars Hofhansl
>             Fix For: 2.0.0, 0.98.9, 0.99.2
>
>         Attachments: 12411-v2.txt, 12411-v3.txt, 12411-v4.txt, 12411.txt
>
>
> In the light of HDFS-6735 we might want to consider refraining from seek + read completely
and only perform preads.
> For example currently a compaction can lock out every other scanner over the file which
the compaction is currently reading for compaction.
> At the very least we can introduce an option to avoid seek + read, so we can allow testing
this in various scenarios.
> This will definitely be of great importance for projects like Phoenix which parallelize
queries intra region (and hence readers will used concurrently by multiple scanner with high
likelihood.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message