Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
MIME-Version: 1.0
Sender: saint.ack@gmail.com
In-Reply-To: 
 <1EFFC0EDF85B044A99A02C925FDA3755200DBBDB@HQITEXCH07.pclc0.merkle.local>
References: <1ccdad83-523d-4621-b386-a59438e482c1@me.com>
	<1EFFC0EDF85B044A99A02C925FDA3755200DBBDB@HQITEXCH07.pclc0.merkle.local>
Date: Tue, 8 Mar 2016 09:28:31 -0800
Message-ID: 
 <CADcMMgEurkZsZ_qd=Z3zmuG3cnng0xjQgXMbDz3+toQ6Uynhqw@mail.gmail.com>
Subject: Re: HBase poor write performance
From: Stack <stack@duboce.net>
To: Hbase-User <user@hbase.apache.org>
Content-Type: multipart/alternative; boundary=001a113e51620709ad052d8ce9f4

--001a113e51620709ad052d8ce9f4
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Tue, Mar 8, 2016 at 8:49 AM, Frank Luo <jluo@merkleinc.com> wrote:

> Akmal,
>
> We have been suffering the issue for two years now without a good
> solution. From what I learned, it is not really a good idea to do heavy
> online hbase puts. The first thing you encounter will be performance caus=
ed
> by compact no matter how you tune parameters. Then later on you will see
> job failures because hbase operation timeouts and/or region server crashe=
s.
>
> Light writes, heavy reads are generally OK.
>
>
What version are you running Frank?

Yes, bulk load is >>> than Puts via API but I'd be interested in what
'light' means for you.

Thanks,
St.Ack


> For heavy puts, the best practice is to prepare tables offline, then turn
> it on for reads.
>
> If online heavy puts not avoidable, you might get the best out of it if
> you manage compact/split by yourself. Meaning when # of files per region
> reaches certain number, stops writing, performs  compacts and splits with
> large regions; then resume writing.
>
> I hope it helps.
>
> Frank Luo
>
> From: Akmal Abbasov [mailto:akmal.abbasov@icloud.com]
> Sent: Tuesday, March 08, 2016 10:29 AM
> To: user@hbase.apache.org
> Subject: HBase poor write performance
>
> Hi,
> I'm testing HBase to choose the right hardware configurations for a heavy
> write use case. I'm testing using YCSB.
> The cluster consist of 2 masters, and 5 regionservers(4 cores, 14GB ram,
> 4x512GB SSD).
> I've created a new table in HBase, presplit it to 50 regions. I'm running
> 3 clients each running 50 threads, to insert data.
> I'm using the default HBase settings. After running few tests, I can see
> that the cluster is underutilized, in fact memory usage is around 30%.
> The main problem I see for now is compactions, compactionQueueLength is
> growing very fast, and compaction process is always running.
> I found that there are hbase.regionserver.thread.compaction.small and
> hbase.regionserver.thread.compaction.large but couldn't find information
> regarding their default values.
> I am also planing to increase the regions number and the memstore size to
> increase utilization of the cluster and performance.
> Which other settings should be tuned to improve both utilization and
> performance?
> Thank you.
>
>
> I'm using HBase 0.98.7 and regionserver heap size is 7GB.
>
>
> Regards, Akmal
>
> This email and any attachments transmitted with it are intended for use b=
y
> the intended recipient(s) only. If you have received this email in error,
> please notify the sender immediately and then delete it. If you are not t=
he
> intended recipient, you must not keep, use, disclose, copy or distribute
> this email without the author=E2=80=99s prior permission. We take precaut=
ions to
> minimize the risk of transmitting software viruses, but we advise you to
> perform your own virus checks on any attachment to this message. We canno=
t
> accept liability for any loss or damage caused by software viruses. The
> information contained in this communication may be confidential and may b=
e
> subject to the attorney-client privilege.
>

--001a113e51620709ad052d8ce9f4--