Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 282E218DC4 for ; Mon, 26 Oct 2015 18:26:03 +0000 (UTC) Received: (qmail 73928 invoked by uid 500); 26 Oct 2015 18:25:28 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 73873 invoked by uid 500); 26 Oct 2015 18:25:28 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 73823 invoked by uid 99); 26 Oct 2015 18:25:27 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 26 Oct 2015 18:25:27 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id BA5E52C14F2 for ; Mon, 26 Oct 2015 18:25:27 +0000 (UTC) Date: Mon, 26 Oct 2015 18:25:27 +0000 (UTC) From: "stack (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (HBASE-14697) Random Read Improvements MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 stack created HBASE-14697: ----------------------------- Summary: Random Read Improvements Key: HBASE-14697 URL: https://issues.apache.org/jira/browse/HBASE-14697 Project: HBase Issue Type: Umbrella Components: Performance Reporter: stack I was looking at something else and then got drawn into graphs of where we are spending time doing random reads. Interesting is that most CPU -- ~5% is in metrics adding -- and bloomfilter look ups cost. This is an umbrella background issue to host random read improvements as subtasks. Here is what perf top shows doing YCSB workloadc (all random read) against a server that is up on a 5-node HDFS cluster hosting 7 regions loaded by YCSB. {code} 4.91% perf-32667.map [.] Lorg/apache/hadoop/metrics2/lib/MutableHistogram;.add 2.27% perf-32667.map [.] Lorg/apache/hadoop/hbase/ipc/RpcServer$Call;.setResponse 2.10% perf-32667.map [.] Lorg/apache/hadoop/hbase/io/hfile/CompoundBloomFilter;.checkContains 2.06% perf-32667.map [.] Lorg/apache/hadoop/hbase/regionserver/HRegion$RegionScannerImpl;. 1.94% perf-32667.map [.] Lorg/apache/hadoop/hbase/ipc/CallRunner;.run 1.89% perf-32667.map [.] Lorg/apache/hadoop/hbase/io/hfile/CombinedBlockCache;.getBlock 1.82% perf-32667.map [.] Lorg/apache/hadoop/hbase/regionserver/RSRpcServices;.get 1.66% perf-32667.map [.] Lorg/apache/hadoop/hbase/util/Counter;.add 1.66% perf-32667.map [.] Lorg/apache/hadoop/hbase/regionserver/StoreFileScanner;.getScannersForStoreFiles 1.41% perf-32667.map [.] Lorg/apache/hadoop/hbase/util/MurmurHash;.hash 1.40% [kernel] [k] _spin_lock 1.34% perf-32667.map [.] Lsun/nio/ch/SocketChannelImpl;.read 1.32% perf-32667.map [.] Ljava/util/concurrent/locks/ReentrantReadWriteLock$ReadLock;.unlock 1.28% perf-32667.map [.] Lorg/apache/hadoop/hbase/regionserver/StoreFileScanner;.shouldUseScanner 1.27% perf-32667.map [.] Lorg/apache/hadoop/hbase/regionserver/StoreFile$Reader;.checkGeneralBloomFilter 1.17% perf-32667.map [.] Lorg/apache/hadoop/hbase/ipc/RpcServer$Connection;.processRequest 1.15% perf-32667.map [.] Lorg/apache/hadoop/hbase/ipc/RpcServer$Listener$Reader;.processing 1.14% perf-32667.map [.] Lsun/nio/ch/SocketChannelImpl;.write 1.13% [vdso] [.] 0x00007fffbedff8d4 1.11% [kernel] [k] copy_user_generic_string 1.11% perf-32667.map [.] Lsun/nio/ch/IOUtil;.write 1.05% [nx_nic] [.] 0x000000005f49014a 0.96% perf-32667.map [.] Ljava/util/concurrent/ConcurrentHashMap;.replaceNode 0.88% perf-32667.map [.] Ljava/util/concurrent/locks/ReentrantReadWriteLock$Sync;.tryAcquireShared 0.86% perf-32667.map [.] Ljava/util/concurrent/ConcurrentHashMap;.putVal 0.82% perf-32667.map [.] Lorg/apache/hadoop/hbase/regionserver/StoreFile$Reader;.passesKeyRangeFilter 0.81% perf-32667.map [.] Lorg/apache/hadoop/hbase/ipc/RpcServer;.call 0.77% perf-32667.map [.] Lorg/apache/hadoop/hbase/protobuf/ProtobufUtil;.toGet 0.76% perf-32667.map [.] Lsun/nio/ch/Util;.getTemporaryDirectBuffer {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)