Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of yuzhihong@gmail.com
 designates 209.85.161.41 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=Cj/4Wiy/Y+cSNIuEoRS0yZzImcHPApGt2XREct4Xa3N6V2Bk4LQVRTUzsE2qUgwOK+
         YUyO2YY1gsAdwnOHDbK6TjajAiBrkQ2chlEkpKdDMxCDeqe2Y3gLq1cZ+ZwM+w+fHjmy
         PCtrldgj8YzFZKvtZQx/boRvQ39rBZ9jurFv4=
MIME-Version: 1.0
In-Reply-To: 
 <5A76F6CE309AD049AAF9A039A39242820F1A5850@sc-mbx04.TheFacebook.com>
References: 
 <996FE584DE9DE642B04DF887487D18F001E34AD0F9@SP2-EX07VS01.ds.corp.yahoo.com>
	<AANLkTindv931jd79O3G8_vJkJxZOXcpkkFp=kiCW9BHa@mail.gmail.com>
	<5A76F6CE309AD049AAF9A039A39242820F1A5850@sc-mbx04.TheFacebook.com>
Date: Thu, 2 Dec 2010 19:44:53 -0800
Message-ID: <AANLkTi=SPbBKp+5tRWY8fxUZ=-b9pG+fV9C2_TfukjZH@mail.gmail.com>
Subject: Re: region, regionserver questions
From: Ted Yu <yuzhihong@gmail.com>
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=20cf30433e327c56b90496795c1d

--20cf30433e327c56b90496795c1d
Content-Type: text/plain; charset=ISO-8859-1

When a table is created with N regions, is it possible to distribute them
(almost) equally among the region servers ?

Thanks

On Thu, Dec 2, 2010 at 3:10 PM, Jonathan Gray <jgray@fb.com> wrote:

> Yeah, I'd recommend just using the normal TIF which will have a map task
> per region, attempts to schedule it on that node, and each task would talk
> to only one (hopefully local) server.
>
> As for assignment, the story has changed significantly between previous
> versions and the upcoming 0.90 release.
>
> In 0.90, there are two modes of startup assignment.  The new default is
> 'retain assignment' where the master will attempt to reuse whatever the last
> set of assignments were on the previous run of the cluster.  The other
> option, if you turn off retain assignment, is round-robin.  This round-robin
> assignment would give you what you want (an approximately equal number of
> regions of each table on each server).
>
> What I've done to get good distribution of the tables is startup with
> round-robin, then from then on use retain assignment.
>
> JG
>
> > -----Original Message-----
> > From: Sean Sechrist [mailto:ssechrist@gmail.com]
> > Sent: Thursday, December 02, 2010 2:50 PM
> > To: user@hbase.apache.org
> > Subject: Re: region, regionserver questions
> >
> > Hey Albert,
> >
> > If you use TableInputFormat, it will create one map task per region in
> that
> > table. So, each mapper should just talk to one regionserver.
> >
> > -Sean
> >
> > On Thu, Dec 2, 2010 at 5:26 PM, Albert Shau <ashau@yahoo-inc.com> wrote:
> >
> > > Hi,
> > >
> > > I'm doing a distributed scan of an hbase table using map-reduce by
> taking
> > > all the regions belonging to a regionserver, and then assigning those
> > > regions to a mapper (so there's 1 mapper per regionserver, and each
> > mapper
> > > only talks to one regionserver).  However, doing it this way I'm
> getting
> > > some data skew.  For example, I have 2 tables U and T.  Each
> regionserver
> > > may have 30 regions, but one regionserver might have 10 regions from
> > table U
> > > while another regionserver might have 25 regions from table U.  Is
> there
> > a
> > > way to balance regions per table per regionserver (so that each
> > regionserver
> > > has 15 regions from table U for example)?  Or should I just not worry
> > about
> > > trying to have each individual mapper only talk to one regionserver?
> > >
> > > Also, how do regions get assigned to regionservers?  Is it based on
> data
> > > locality?  Region start/end keys?  Randomly?
> > >
> > > Thanks,
> > > Albert
> > >
>

--20cf30433e327c56b90496795c1d--