hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-7868) HFile performance regression between 0.92 and 0.94
Date Mon, 18 Feb 2013 16:37:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-7868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13580702#comment-13580702
] 

Lars Hofhansl commented on HBASE-7868:
--------------------------------------

JM, what I am saying is that
{{bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=100000 filterScan 1}}
opens 100000 scanners and calls next until there are no more rows. That would "never" finish
unless your each scan returns almost no data. You are not testing scan performance with that,
but RPC/Network performance (if you're testing locally you're testing context switch performance
through the lo interface).

Do you seed the table with by calling SequentialWriteTest before running each read test?
(I find that I have to do that, and otherwise there is just an empty test table)

The performance improvement after 0.94.0 that I measure this way is in line with other tests
I did before.
                
> HFile performance regression between 0.92 and 0.94
> --------------------------------------------------
>
>                 Key: HBASE-7868
>                 URL: https://issues.apache.org/jira/browse/HBASE-7868
>             Project: HBase
>          Issue Type: Bug
>          Components: io
>    Affects Versions: 0.94.5
>            Reporter: Matteo Bertozzi
>            Assignee: Matteo Bertozzi
>             Fix For: 0.94.6
>
>         Attachments: FilteredScan.png, hfileperf-graphs.png, performances.pdf, performances.pdf
>
>
> By HFilePerformanceEvaluation seems that 0.94 is slower then 0.92
> Looking at the profiler for the Scan path, seems that most of the time, compared to 92,
is spent in the metrics dictionary lookup. [~eclark] pointed out the new per family/block
metrics.
> By commenting the metrics call in HFileReaderV2, the performance seems to get better,
but maybe metrics is not the only problem.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message