Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of jheitmann@gmail.com
 designates 74.125.83.169 as permitted sender)
Content-Type: text/plain; charset=iso-8859-1
Mime-Version: 1.0 (Apple Message framework v1244.3)
Subject: Re: Region "auto-split"
From: John Heitmann <jheitmann@gmail.com>
In-Reply-To: 
 <CADcMMgFGYqC9vaSR2KJYGNO5xvBY-g4KtxcfPky3gzeyVEOtTw@mail.gmail.com>
Date: Mon, 11 Jul 2011 11:07:12 -0700
Content-Transfer-Encoding: quoted-printable
Message-Id: <024452DA-2F3A-4E74-9E64-FDA9D5FAC91D@gmail.com>
References: 
 <CAHdy+0-TknFYBC-G3iGhP3B66SfF_A++3c3qSYBK1WgyuGADGA@mail.gmail.com>
 <30D4476CB62BAE4589FD646476C29CD20167E7EA@SZXEML505-MBX.china.huawei.com>
 <CAHdy+09xD8aGc11Uemh91gsyOj8+49f4c+8xb+OzUwO7WK3x0Q@mail.gmail.com>
 <CADcMMgFGYqC9vaSR2KJYGNO5xvBY-g4KtxcfPky3gzeyVEOtTw@mail.gmail.com>
To: dev@hbase.apache.org

Apologies for the "me too" message, but: me too. I've been seeing this =
for a month or two on my trunk-based setup. I looked for log messages, =
but things looked pretty silent, and when I looked at the code my =
impression was identical to Michael's in that I didn't see a plausible =
path for it to work outside of extreme circumstances.=20

Now that I look at it again, it seems like auto-splitting is initiated =
only if 1) There are too many store files per region and 2) The max =
blocking time hasn't been exceeded. It looks like the requestSplit call =
should be hoisted a bit in MemStoreFlusher.flushRegion().

John

On Jul 11, 2011, at 9:17 AM, Stack wrote:

> If you have an individual storefile > 256M and its not splitting
> something is wrong.  Check the logs on the regionserver carrying your
> single region.  See if any clue therein.
>=20
> St.Ack
>=20
>=20
> On Mon, Jul 11, 2011 at 5:54 AM, Michael Morello
> <michael.morello@gmail.com> wrote:
>> Hi,
>>=20
>> I use the default values, so hbase.hregion.max.filesize should be 256
>> Mo as defined in HConstants.
>>=20
>> Regards,
>>=20
>> 2011/7/11 Zhoushuaifeng <zhoushuaifeng@huawei.com>:
>>> What's your settings of this : hbase.hregion.max.filesize ?
>>> If your setting is more than 1GB, and total size in the region less =
than this , your region will not split.
>>>=20
>>=20
>>> -----Original Message-----
>>> From: Michael Morello [mailto:michael.morello@gmail.com]
>>> Sent: Monday, July 11, 2011 8:28 PM
>>> To: dev@hbase.apache.org
>>> Subject: Region "auto-split"
>>>=20
>>> Hello,
>>>=20
>>> I'm testing the trunk revision of HBase and there is something that
>>> seems strange to me regarding the "auto-split" region feature, here =
is
>>> my test case :
>>> I generate a lot of data with a simple client into a newly created
>>> table with no predefined region. Everything is stored in the first
>>> "default" region but this region is never splitted (I currently have =
a
>>> region with a single file store of more than 1Go).
>>>=20
>>> It seems that it is the responsability of the MemStoreFlusher to =
call
>>> the CompactSplitThread (line 359 of MemStoreFlusher) and schedule a
>>> split, but this part is never called.
>>> More precisely one of the condition that is expected to be true in
>>> order to schedule a split is that isTooManyStoreFiles(region) is =
true,
>>> this could never happen.
>>>=20
>>> Is it an expected behaviour or is it a known problem ?
>>=20