Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of shujamughal@gmail.com
 designates 74.125.82.41 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=b+OF/2Co04ZUnAnbgpllz1VhG5TAiaTZUEjbMYDcjd7O8m0NHIt10S6md7VOgsVTz0
         HW0vwDbrFgOtJwV+T1wgzwd8uiqSIo0QZQzJ/ZrQMFkQZFDz/Jz+jQX1lWF3CYkKStyr
         xDEFo/uMBDjPLgyLeSzvEaN8mnHO9Eqrr8F+U=
MIME-Version: 1.0
In-Reply-To: <AANLkTikCWz6yy8xHtN7YsNGk3hJ=t29BP0HnbqgwggMK@mail.gmail.com>
References: <AANLkTikarixph4-x3hhbszNPYR6FLbQizKAvqe+k4Wd6@mail.gmail.com>
	<AANLkTi=tCUtou4DLdKQU2bR3vaEHaRG1EQ5ANqHF2su_@mail.gmail.com>
	<AANLkTim6_N1pEJXVcdpufoJre+rLZ2xtGpcyj-=j7EdK@mail.gmail.com>
	<AANLkTikRMJsbY5LiACDQR62FJL66MKY=D0kuV2__aNHR@mail.gmail.com>
	<AANLkTimB5z4qevLhYHs-o_0PRt5pnVApiLxLA=_OmuL8@mail.gmail.com>
	<AANLkTi=heK_EiC3Vnhf2MNw84zmPed+WaDqaHv=cxF3e@mail.gmail.com>
	<AANLkTikq_dciOLzKpJLsQT6MUCVE8WA8BusnL8uNwoac@mail.gmail.com>
	<AANLkTi=aeJ56L+cMGktZbUXoYPDsY2s4FOt-bNH8+3v2@mail.gmail.com>
	<AANLkTim=LE-KLsEMqg_awi-8zH_e_Q7J5PHUizNb_bzR@mail.gmail.com>
	<4CDD8328.9030007@opendns.com>
	<AANLkTikCWz6yy8xHtN7YsNGk3hJ=t29BP0HnbqgwggMK@mail.gmail.com>
Date: Mon, 15 Nov 2010 13:24:37 +0500
Message-ID: <AANLkTi=Bz5_G-p+tr=u=iiwdA66aKrPcLjGruZS6qdJt@mail.gmail.com>
Subject: Re: Bulk Load Sample Code
From: Shuja Rehman <shujamughal@gmail.com>
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=001636426735c30cbe0495132b16

--001636426735c30cbe0495132b16
Content-Type: text/plain; charset=ISO-8859-1

If HRegionPartitioner works correctly then what is the use of
configureIncrementalLoad() as discussed here

http://hbase.apache.org/docs/r0.89.20100621/bulk-loads.html

and why this link did not discuss about HRegionPartitioner? Where is the
documentaion of HRegionPartitioner and in which case we give preference to
HRegionPartitioner  over configureIncrementalLoad(). What is the difference
in both?


On Sat, Nov 13, 2010 at 5:12 AM, Todd Lipcon <todd@cloudera.com> wrote:

> I'm surprised that HRegionPartitioner works correctly for incremental load.
> It definitely won't work if the regions are also shifting during the MR
> job.
>
> Thanks
> -Todd
>
> On Fri, Nov 12, 2010 at 10:10 AM, Adam Phelps <amp@opendns.com> wrote:
>
> > On 11/10/10 11:57 AM, Stack wrote:
> >
> >> On Wed, Nov 10, 2010 at 11:53 AM, Shuja Rehman<shujamughal@gmail.com>
> >>  wrote:
> >>
> >>> oh! I think u have not read the full post. The essay has 3 paragraphs
>  :)
> >>>
> >>> *Should I need to add the following line also
> >>>
> >>>  job.setPartitionerClass(TotalOrderPartitioner.class);
> >>>
> >>>
> >> You need to specify other than default partitioner so yes, above seems
> >> necessary (Be aware that if only one reducer, all may appear to work
> >> though your partitioner is bad... its when you have multiple reducers
> >> that bad partitioner will show).
> >>
> >
> > I skimmed over this thread as we've been using LoadIncrementalHFiles to
> > load the output of our MR jobs, however it looks like we're using
> > HRegionPartitioner rather than TotalOrderPartitioner.  The current code
> is
> > definitely working, however the page regarding bulk loads that was posted
> > earlier implies that TotalOrderPartitioner is best for efficiency.  What
> is
> > the difference between the two?
> >
> > - Adam
> >
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>


-- 
Regards
Shuja-ur-Rehman Baig
<http://pk.linkedin.com/in/shujamughal>

--001636426735c30cbe0495132b16--