Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of stuart.white1@gmail.com
 designates 209.85.217.215 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=B9ORqcwEuUQGrld+YeGJrTBcuJNitIYjo8PRt1tG6k1ROgp62kl2vJMxyyYK5EQtkk
         zxFE+vlPCcqyMQ7KzSQiiBM7GYoq6jFQbLWKL816QiM0orijAWlHDzpCuGwAi/DqTuve
         mkd1nDQVRq0bCrDNWFwD4Jl+3iHsHYZnfQOIs=
MIME-Version: 1.0
In-Reply-To: <4af5cd780906270825w1242a689p96d23e293d2cedb0@mail.gmail.com>
References: <4af5cd780906270825w1242a689p96d23e293d2cedb0@mail.gmail.com>
Date: Sat, 27 Jun 2009 10:30:17 -0500
Message-ID: <4af5cd780906270830r58d4d009r5e5ba258629e163b@mail.gmail.com>
Subject: Re: Confused about partitioning and reducers
From: Stuart White <stuart.white1@gmail.com>
To: core-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=001e680f102056e976046d5622fe

--001e680f102056e976046d5622fe
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Please disregard this question.  I think I'm mistaken.

On Sat, Jun 27, 2009 at 10:25 AM, Stuart White <stuart.white1@gmail.com>wrote:

> If I call HashPartitioner.getPartition(), passing a key of 4 and a
> numPartitions of 5, it returns a partition of 4.  (Which is what I would
> expect.)
>
> However, if I have a mapred job, and in my mapper I emit a record with key
> 4, I'm configured to use the HashPartitioner, I have 5 Reducers configured,
> and I'm using the IdentityReducer, the record with key 4 gets handled by
> Reducer #0 (because it gets written out to part-00000).
>
> I would have expected a record with key 4 to be handled by reducer #4 (and
> therefore written to part-00004) because the HashPartitioner returns 4 for a
> key of 4 and a numPartitions of 5.
>
> Obviously I'm missing something here.  What is the logic for deciding which
> partition of records is handled by which reducer instance?
>
> It can't be random, otherwise mapside join wouldn't work.
>
> Thanks.
>

--001e680f102056e976046d5622fe--