hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Rodionov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14697) Random Read Improvements
Date Mon, 26 Oct 2015 18:35:27 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14974790#comment-14974790
] 

Vladimir Rodionov commented on HBASE-14697:
-------------------------------------------

No obvious hotspot. Where is the code which look ups key in a block? It is a sequential scan.

> Random Read Improvements
> ------------------------
>
>                 Key: HBASE-14697
>                 URL: https://issues.apache.org/jira/browse/HBASE-14697
>             Project: HBase
>          Issue Type: Umbrella
>          Components: Performance
>            Reporter: stack
>         Attachments: flamegraph-32667.svg
>
>
> I was looking at something else and then got drawn into graphs of where we are spending
time doing random reads. Interesting is that most CPU -- ~5% is in metrics adding -- and bloomfilter
look ups cost.
> This is an umbrella background issue to host random read improvements as subtasks.
> Here is what perf top shows doing YCSB workloadc (all random read) against a server that
is up on a 5-node HDFS cluster hosting 7 regions loaded by YCSB.
> {code}
>   4.91%  perf-32667.map      [.] Lorg/apache/hadoop/metrics2/lib/MutableHistogram;.add
>   2.27%  perf-32667.map      [.] Lorg/apache/hadoop/hbase/ipc/RpcServer$Call;.setResponse
>   2.10%  perf-32667.map      [.] Lorg/apache/hadoop/hbase/io/hfile/CompoundBloomFilter;.checkContains
>   2.06%  perf-32667.map      [.] Lorg/apache/hadoop/hbase/regionserver/HRegion$RegionScannerImpl;.<init>
>   1.94%  perf-32667.map      [.] Lorg/apache/hadoop/hbase/ipc/CallRunner;.run
>   1.89%  perf-32667.map      [.] Lorg/apache/hadoop/hbase/io/hfile/CombinedBlockCache;.getBlock
>   1.82%  perf-32667.map      [.] Lorg/apache/hadoop/hbase/regionserver/RSRpcServices;.get
>   1.66%  perf-32667.map      [.] Lorg/apache/hadoop/hbase/util/Counter;.add
>   1.66%  perf-32667.map      [.] Lorg/apache/hadoop/hbase/regionserver/StoreFileScanner;.getScannersForStoreFiles
>   1.41%  perf-32667.map      [.] Lorg/apache/hadoop/hbase/util/MurmurHash;.hash
>   1.40%  [kernel]            [k] _spin_lock
>   1.34%  perf-32667.map      [.] Lsun/nio/ch/SocketChannelImpl;.read
>   1.32%  perf-32667.map      [.] Ljava/util/concurrent/locks/ReentrantReadWriteLock$ReadLock;.unlock
>   1.28%  perf-32667.map      [.] Lorg/apache/hadoop/hbase/regionserver/StoreFileScanner;.shouldUseScanner
>   1.27%  perf-32667.map      [.] Lorg/apache/hadoop/hbase/regionserver/StoreFile$Reader;.checkGeneralBloomFilter
>   1.17%  perf-32667.map      [.] Lorg/apache/hadoop/hbase/ipc/RpcServer$Connection;.processRequest
>   1.15%  perf-32667.map      [.] Lorg/apache/hadoop/hbase/ipc/RpcServer$Listener$Reader;.processing
>   1.14%  perf-32667.map      [.] Lsun/nio/ch/SocketChannelImpl;.write
>   1.13%  [vdso]              [.] 0x00007fffbedff8d4
>   1.11%  [kernel]            [k] copy_user_generic_string
>   1.11%  perf-32667.map      [.] Lsun/nio/ch/IOUtil;.write
>   1.05%  [nx_nic]            [.] 0x000000005f49014a
>   0.96%  perf-32667.map      [.] Ljava/util/concurrent/ConcurrentHashMap;.replaceNode
>   0.88%  perf-32667.map      [.] Ljava/util/concurrent/locks/ReentrantReadWriteLock$Sync;.tryAcquireShared
>   0.86%  perf-32667.map      [.] Ljava/util/concurrent/ConcurrentHashMap;.putVal
>   0.82%  perf-32667.map      [.] Lorg/apache/hadoop/hbase/regionserver/StoreFile$Reader;.passesKeyRangeFilter
>   0.81%  perf-32667.map      [.] Lorg/apache/hadoop/hbase/ipc/RpcServer;.call
>   0.77%  perf-32667.map      [.] Lorg/apache/hadoop/hbase/protobuf/ProtobufUtil;.toGet
>   0.76%  perf-32667.map      [.] Lsun/nio/ch/Util;.getTemporaryDirectBuffer
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message