Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of amits@infolinks.com
 designates 207.126.144.141 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAORosc=aPAyhoZYyiQbuyp0SN32+8=H+GZK0AKnmRAk8gpz-Fw@mail.gmail.com>
References: 
 <CAORosc=aPAyhoZYyiQbuyp0SN32+8=H+GZK0AKnmRAk8gpz-Fw@mail.gmail.com>
Date: Mon, 28 Jan 2013 13:51:33 +0200
Message-ID: 
 <CAAMYKhpUe5Rc0QR0KR+vO_AkfbZtYPcMw2ZSeMe9fLJ1jeRehw@mail.gmail.com>
Subject: Re: Pre-split Region Boundaries
From: Amit Sela <amits@infolinks.com>
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=e0cb4efe2a640cd37a04d457e67a

--e0cb4efe2a640cd37a04d457e67a
Content-Type: text/plain; charset=ISO-8859-1

We are pre-splitting our tables before bulk loading also but we don't use
the RegionSplitter.

We split manually (we did some testing and found the optimal split points)
by putting into .META table a new HRegionInfo, assigning that region
(HBaseAdmin.assign("region name")) and after you finish assigning all the
regions don't forget to clear the region cache.

I know it's a little bit "intrusive" but it works for us.

On Fri, Jan 25, 2013 at 4:45 PM, Rob Styles <rob@dynamicorange.com> wrote:

> Hi,
>
> I'm tuning hbase for storage of a few billion rows and, more or less, bulk
> loading.
>
> I'm using MD5 strings as row ids to create an evenly distributed range and
> non-sequential values during loading and this is working relatively well
> for us.
>
> I've pre-split my tables using org.apache.hadoop.hbase.util.RegionSplitter
> from the command line and had expected it to create regions covering 00000
> - fffff as per the docs. My regions come out different though, before
> loading any data.
>
> With 200 regions the first region ends with 00a3d70a and the regions go up
> from there. The last region has a start key of 7f5c28c6 which is only
> half-way through the address space. This means my last region gets hot
> during loading.
>
> I know I must have missed something but not sure what. Any help greatly
> appreciated.
>
> thanks
>
> rob
>

--e0cb4efe2a640cd37a04d457e67a--