Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: error (athena.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: 
 <DC5EBE7F3610EB4CA5C7E92D76873E1517ECF1DCC4@exchange2007.carrieriq.com>
References: 
 <DC5EBE7F3610EB4CA5C7E92D76873E1517ECF1DCC2@exchange2007.carrieriq.com>
 <CAPQV63VWJxs6KHAvBs4aVcRLB=qg5cvX-cb+T1a-UfGArrZK2Q@mail.gmail.com>
 <DC5EBE7F3610EB4CA5C7E92D76873E1517ECF1DCC3@exchange2007.carrieriq.com>
 <DC5EBE7F3610EB4CA5C7E92D76873E1517ECF1DCC4@exchange2007.carrieriq.com>
From: Jean-Marc Spaggiari <jean-marc@spaggiari.org>
Date: Sun, 28 Jul 2013 20:50:20 -0400
Message-ID: 
 <CAPQV63XJzvnbFZAM=HCXcp_gnJ8Vba9JWksObZdVYZgThnsXcw@mail.gmail.com>
Subject: Re: MAX_FILESIZE and hbase.hregion.max.filesize are both 10Gb
To: dev@hbase.apache.org
Content-Type: multipart/alternative; boundary=047d7b343b3aa9fbfd04e29be1a4

--047d7b343b3aa9fbfd04e29be1a4
Content-Type: text/plain; charset=UTF-8

"Yes it works, of course." It's not working for me ;) so was not sure.

It's normal to have regions under the half of the MAX_FILESIZE. When a
regions is more than MAX_FILESIZE it's splitted in 2. So one can be more,
and the other one can be less.

I will say, average 5GB will have been a good value, but even 3.6 is still
not so bad.

Have you pre-splitted the regions initially? Is it possible that you have
not-used pre-splitted regions?

You can you Hannibal to have a quick view of what the sizes are

JM

2013/7/28 Vladimir Rodionov <vrodionov@carrieriq.com>

> The final stats:
>
> Total HDFS size - 376GB
> #regions: 109 - avg. region size ~ 3.6GB
>
> Something is wrong here. I expected fewer regions. The regions get split
> at sizes much lower than
> hbase.hregion.max.filesize and/or  MAX_FILESIZE.
>
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: vrodionov@carrieriq.com
>
> ________________________________________
> From: Vladimir Rodionov
> Sent: Sunday, July 28, 2013 3:35 PM
> To: dev@hbase.apache.org
> Subject: RE: MAX_FILESIZE and hbase.hregion.max.filesize are both 10Gb
>
> Yes it works, of course.
>
> Its in original post - ~ 10gB
>
> <property>
> <name>hbase.hregion.max.filesize</name>
>    <value>10737418240</value>
>    <source>hbase-site.xml</source>
> </property>
>
>
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: vrodionov@carrieriq.com
>
> ________________________________________
> From: Jean-Marc Spaggiari [jean-marc@spaggiari.org]
> Sent: Sunday, July 28, 2013 2:30 PM
> To: dev@hbase.apache.org
> Subject: Re: MAX_FILESIZE and hbase.hregion.max.filesize are both 10Gb
>
> Hi Vladimir,
>
> Is this link working for you? http://MASTERURL:60010/conf ? If yes, what
> do
> you have for hbase.hregion.max.filesize? To make sure the property below is
> considerered.
>
> For the table config, did you get it from the webui?
>
> JM
>
> 2013/7/28 Vladimir Rodionov <vrodionov@carrieriq.com>
>
> > but all regions keep getting split at 1Gb
> >
> > I have 71 regions and 70GB of data in 'usertable' despite the fact that:
> >
> > table config is:
> > {NAME => 'usertable', DEFERRED_LOG_FLUSH => 'true', MAX_FILESIZE =>
> > '10000000000', FAMILIES => [{NAME => 'cf', BLOOMFILTER => 'ROWCOL',
> > VERSIONS => '1', COMPRESSION => 'GZ'}]}
> >
> > and hbase-size.xml has the following config:
> >
> > <property>
> >    <name>hbase.hregion.max.filesize</name>
> >    <value>10737418240</value>
> >    <source>hbase-site.xml</source>
> > </property>
> >
> > HBase 0.94.6
> >
> > Best regards,
> > Vladimir Rodionov
> > Principal Platform Engineer
> > Carrier IQ, www.carrieriq.com
> > e-mail: vrodionov@carrieriq.com
> >
> > Confidentiality Notice:  The information contained in this message,
> > including any attachments hereto, may be confidential and is intended to
> be
> > read only by the individual or entity to whom this message is addressed.
> If
> > the reader of this message is not the intended recipient or an agent or
> > designee of the intended recipient, please note that any review, use,
> > disclosure or distribution of this message or its attachments, in any
> form,
> > is strictly prohibited.  If you have received this message in error,
> please
> > immediately notify the sender and/or Notifications@carrieriq.com and
> > delete or destroy any copy of this message and its attachments.
> >
>
> Confidentiality Notice:  The information contained in this message,
> including any attachments hereto, may be confidential and is intended to be
> read only by the individual or entity to whom this message is addressed. If
> the reader of this message is not the intended recipient or an agent or
> designee of the intended recipient, please note that any review, use,
> disclosure or distribution of this message or its attachments, in any form,
> is strictly prohibited.  If you have received this message in error, please
> immediately notify the sender and/or Notifications@carrieriq.com and
> delete or destroy any copy of this message and its attachments.
>

--047d7b343b3aa9fbfd04e29be1a4--