Return-Path: 
 <user-return-34956-apmail-hbase-user-archive=hbase.apache.org@hbase.apache.org>
X-Original-To: apmail-hbase-user-archive@www.apache.org
Delivered-To: apmail-hbase-user-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id D53C7FBE6
	for <apmail-hbase-user-archive@www.apache.org>;
 Fri, 22 Mar 2013 14:44:47 +0000 (UTC)
Received: (qmail 5041 invoked by uid 500); 22 Mar 2013 14:44:45 -0000
Delivered-To: apmail-hbase-user-archive@hbase.apache.org
Received: (qmail 4987 invoked by uid 500); 22 Mar 2013 14:44:45 -0000
Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@hbase.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@hbase.apache.org>
List-Post: <mailto:user@hbase.apache.org>
List-Id: <user.hbase.apache.org>
Reply-To: user@hbase.apache.org
Delivered-To: mailing list user@hbase.apache.org
Received: (qmail 4936 invoked by uid 99); 22 Mar 2013 14:44:43 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 22 Mar 2013 14:44:43 +0000
X-ASF-Spam-Status: No, hits=1.5 required=5.0
	tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of tarang.dawer@gmail.com
 designates 209.85.212.176 as permitted sender)
Received: from [209.85.212.176] (HELO mail-wi0-f176.google.com)
 (209.85.212.176)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 22 Mar 2013 14:44:38 +0000
Received: by mail-wi0-f176.google.com with SMTP id hm14so7911548wib.9
        for <user@hbase.apache.org>; Fri, 22 Mar 2013 07:44:18 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:x-received:in-reply-to:references:date:message-id
         :subject:from:to:content-type;
        bh=ik6ovYm1rBeglQh6bXPeGGvapcNiCM0GwTEZktOlhag=;
        b=ZbE0P8OfEGJF57vDC/sipsIp+THg8JbPwrePnuYS3AFae8g18HEZe6GX9ygOIZk8U3
         a16vgS70W4NISxYDPOmnZAxXcO16dsXB7/Prkwhmk4sLS4WtHIWPXk54ae4hLe9RBW65
         1hgfcOcNCAgS5KpP6CZSR/KpcErTH1dvCvTdmkMHUUrqIuNxhHkOEaoHG5bPfEqQP05L
         rdA1X6LpD41Vk4nxgKEIL1c6R9qKxrU+IvmAGFwXaU4VAGplYcUdQSSLZcDIONXpFvES
         i3EKhUMfb7eXI1EOnPIT+706vup3ZAcewg9/xy5im84glyHTzsmBrzSjkgDmzEgOPLj+
         Gz3A==
MIME-Version: 1.0
X-Received: by 10.194.235.196 with SMTP id uo4mr3596648wjc.30.1363963457939;
 Fri, 22 Mar 2013 07:44:17 -0700 (PDT)
Received: by 10.180.182.170 with HTTP; Fri, 22 Mar 2013 07:44:17 -0700 (PDT)
In-Reply-To: 
 <CAPQV63WQF_CxfzPqtOjTsz5uUg7P0bL2yHRfC2fXBKUPyQ21-Q@mail.gmail.com>
References: 
 <CAEGWFQWXi72dho0NAYGHrLGkjXUatUdLn2JnZY0zA=wLZnJGxA@mail.gmail.com>
	<CAPQV63WQF_CxfzPqtOjTsz5uUg7P0bL2yHRfC2fXBKUPyQ21-Q@mail.gmail.com>
Date: Fri, 22 Mar 2013 20:14:17 +0530
Message-ID: 
 <CAEGWFQXLXW7OvOtPh6wZ_B27fnep=cw-B3uELczM8sEFnXhFUw@mail.gmail.com>
Subject: Re: Compaction problem
From: tarang dawer <tarang.dawer@gmail.com>
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=089e01419d7c65b00304d8847dcd
X-Virus-Checked: Checked by ClamAV on apache.org

--089e01419d7c65b00304d8847dcd
Content-Type: text/plain; charset=ISO-8859-1

Hi
As per my use case , I have to write around 100gb data , with a ingestion
speed of around 200 mbps. While writing , i am getting a performance hit by
compaction , which adds to the delay.
I am using a 8 core machine with 16 gb RAM available., 2 Tb hdd 7200RPM.
Got some idea from the archives and  tried pre splitting the regions ,
configured HBase with following parameters(configured the parameters in a
haste , so please guide me if anything's out of order) :-


        <property>
                <name>hbase.hregion.memstore.block.multiplier</name>
                <value>4</value>
        </property>
        <property>
                 <name>hbase.hregion.memstore.flush.size</name>
                 <value>1073741824</value>
        </property>

        <property>
                <name>hbase.hregion.max.filesize</name>
                <value>1073741824</value>
        </property>
        <property>
                <name>hbase.hstore.compactionThreshold</name>
                <value>5</value>
        </property>
        <property>
              <name>hbase.hregion.majorcompaction</name>
                  <value>0</value>
        </property>
        <property>
                <name>hbase.hstore.blockingWaitTime</name>
                <value>30000</value>
        </property>
         <property>
                 <name>hbase.hstore.blockingStoreFiles</name>
                 <value>200</value>
         </property>

  <property>
        <name>hbase.regionserver.lease.period</name>
        <value>3000000</value>
  </property>


but still m not able to achieve the optimal rate , getting around 110 mbps.
Need some optimizations ,so please could you help out ?

Thanks
Tarang Dawer





On Fri, Mar 22, 2013 at 6:05 PM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:

> Hi Tarang,
>
> I will recommand you to take a look at the list archives first to see
> all the discussions related to compaction. You will found many
> interesting hints and tips.
>
>
> http://search-hadoop.com/?q=compactions&fc_project=HBase&fc_type=mail+_hash_+user
>
> After that, you will need to provide more details regarding how you
> are using HBase and how the compaction is impacting you.
>
> JM
>
> 2013/3/22 tarang dawer <tarang.dawer@gmail.com>:
> > Hi
> > I am using HBase 0.94.2 currently. While using it  , its write
> performance,
> > due to compaction is being affeced by compaction.
> > Please could you suggest some quick tips in relation to how to deal with
> it
> > ?
> >
> > Thanks
> > Tarang Dawer
>

--089e01419d7c65b00304d8847dcd--