hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13291) Lift the scan ceiling
Date Sat, 21 Mar 2015 00:02:38 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14372347#comment-14372347

Andrew Purtell commented on HBASE-13291:

A question about this though:
export HBASE_OPTS=" -agentpath:/home/stack/lightweight-java-profiler-read-only/build-64/liblagent.so
-XX:+UseConcMarkSweepGC -XX:+UnlockDiagnosticVMOptions -XX:+PrintInlining -XX:+PrintCompilation
-XX:+UnlockCommercialFeatures -XX:+FlightRecorder "
Probably should break that up. Maybe you did? 

# Use flight recorder to get event trace data for profiling, IO, monitors, etc. 
# Use the lightweight profiler to get trace files for flame graph visualization
# PrintInlining to get a sense of hot methods and suboptimal inlining
# PrintCompilation with hsdis installed into jre/lib/<plat> or a fastdebug JDK build.

I wouldn't combine them, they'll work at various cross-purposes. Maybe not terribly, but results
will be cleaner IMHO

> Lift the scan ceiling
> ---------------------
>                 Key: HBASE-13291
>                 URL: https://issues.apache.org/jira/browse/HBASE-13291
>             Project: HBase
>          Issue Type: Improvement
>          Components: Scanners
>    Affects Versions: 1.0.0
>            Reporter: stack
>            Assignee: stack
>         Attachments: 13291.inlining.txt, q (1).png, traces.7.svg, traces.filterall.svg,
traces.nofilter.svg, traces.smaller.svg
> Scanning medium sized rows with multiple concurrent scanners exhibits interesting 'ceiling'
properties. A server runs at about 6.7k ops a second using 450% of possible 1600% of CPUs
 when 4 clients each with 10 threads doing scan 1000 rows.  If I add '--filterAll' argument
(do not return results), then we run at 1450% of possible 1600% possible but we do 8k ops
a second.
> Let me attach flame graphs for two cases. Unfortunately, there is some frustrating dark
art going on. Let me try figure it... Filing issue in meantime to keep score in.

This message was sent by Atlassian JIRA

View raw message