Return-Path: X-Original-To: apmail-hbase-user-archive@www.apache.org Delivered-To: apmail-hbase-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C38C511A0A for ; Thu, 24 Apr 2014 19:03:02 +0000 (UTC) Received: (qmail 41445 invoked by uid 500); 24 Apr 2014 19:02:58 -0000 Delivered-To: apmail-hbase-user-archive@hbase.apache.org Received: (qmail 41377 invoked by uid 500); 24 Apr 2014 19:02:57 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Received: (qmail 41225 invoked by uid 99); 24 Apr 2014 19:02:57 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 24 Apr 2014 19:02:57 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of Shaosu.Liu@turn.com designates 69.12.158.153 as permitted sender) Received: from [69.12.158.153] (HELO webmail.turn.com) (69.12.158.153) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 24 Apr 2014 19:02:52 +0000 Received: from TURN-MAIL-BK.turn.corp ([169.254.2.117]) by cas.turn.corp ([169.254.161.63]) with mapi id 14.02.0247.003; Thu, 24 Apr 2014 12:02:31 -0700 From: Shaosu Liu To: "user@hbase.apache.org" Subject: concurrent scan optimization Thread-Topic: concurrent scan optimization Thread-Index: Ac9f65FiJyBP6OO4RBG8WYch9oHwOA== Date: Thu, 24 Apr 2014 19:02:30 +0000 Message-ID: <4AA573D518818D458ACF2640C65A2699885AC5A1@turn-mail-bk.turn.corp> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [172.16.115.116] x-tm-as-product-ver: SMEX-11.0.0.1191-7.500.1017-20654.001 x-tm-as-result: No--23.349800-8.000000-31 x-tm-as-user-approved-sender: No x-tm-as-user-blocked-sender: No Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Virus-Checked: Checked by ClamAV on apache.org Hi,=0A= =0A= I want to large amount of concurrent scan (10k), each fetching 200 - 500 ke= ys, each key is around 500 - 1k bytes.=0A= =0A= I have 5 regions servers using offheap cache, data is fully cached and all = machines are on the same rack with gigabit connection. So GC or disk should= not slow down me much. =0A= =0A= Scan caching is set to 10000, each region server has 200 handlers.=0A= =0A= Currently it takes around 30 seconds to process 10k scan concurrently. If I= use asynchbase, it takes around 18seconds. It should be able to possible d= o this around 1 - 2 seconds. =0A= =0A= Could anybody shed light on what I am doing wrong here?=0A= =0A= Thanks,=0A= ~Shaosu Liu=0A=