Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of vrodionov@carrieriq.com
 designates 204.235.122.16 as permitted sender)
From: Vladimir Rodionov <vrodionov@carrieriq.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>
Date: Sun, 23 Mar 2014 12:51:52 -0700
Subject: RE: Effect of region size on compaction performance
Thread-Topic: Effect of region size on compaction performance
Thread-Index: Ac9GO4cqitD2iGeUT3SxwGpb6CJCRAAlc7Af
Message-ID: 
 <DC5EBE7F3610EB4CA5C7E92D76873E1518629B5924@exchange2007.carrieriq.com>
References: 
 <CAE24rAeUt1j57H0t9vwyMToHNgLrAFgEx4SELrZi+6WbDVLRPw@mail.gmail.com>
In-Reply-To: 
 <CAE24rAeUt1j57H0t9vwyMToHNgLrAFgEx4SELrZi+6WbDVLRPw@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

How small is small and how large is large?
Recommended region size is usually between 5-10GB. Too small regions result=
s in more frequent flushes/compactions
and have additional overhead in RS RAM.

>>I am thinking about extending TableInputFormat to override the
>>1-map-per-region default policy as an alternative.

This looks better approach.

Best regards,
Vladimir Rodionov
Principal Platform Engineer
Carrier IQ, www.carrieriq.com
e-mail: vrodionov@carrieriq.com

________________________________________
From: David Koch [ogdude@googlemail.com]
Sent: Saturday, March 22, 2014 6:58 PM
To: user@hbase.apache.org
Subject: Effect of region size on compaction performance

Hello,

We run M/Rs over several HBase tables at the same time and chose to reduce
region sizes in order to make map tasks faster and improve map-slot
turnaround between the concurrent jobs. However, I am worried many regions
will cause longer overall compactions of the HBase data. Is this the case?

I am thinking about extending TableInputFormat to override the
1-map-per-region default policy as an alternative.

Regards,

/David

Confidentiality Notice:  The information contained in this message, includi=
ng any attachments hereto, may be confidential and is intended to be read o=
nly by the individual or entity to whom this message is addressed. If the r=
eader of this message is not the intended recipient or an agent or designee=
 of the intended recipient, please note that any review, use, disclosure or=
 distribution of this message or its attachments, in any form, is strictly =
prohibited.  If you have received this message in error, please immediately=
 notify the sender and/or Notifications@carrieriq.com and delete or destroy=
 any copy of this message and its attachments.