Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of mailinglists19@gmail.com
 designates 72.14.220.152 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=H/2SyvcdHmzXGwBL/pYIWKw5G74cz4Sv36vrXeq3VNxPhdjbQsWVOYyIvrtZXCstCO
         OdpRpRJ+dTT4IdHrb50FQSEhAnuSeAEKpRDx3dH7WrSZ8YqFnqfAoupGASaSFBTO97s7
         0K944TJzDW0dwUAyFJh0HoMFLlk7u4vBYZaFQ=
MIME-Version: 1.0
In-Reply-To: <895334.52215.qm@web32203.mail.mud.yahoo.com>
References: <1b3669280911161108p4fbe9d20i12dd564a121aecac@mail.gmail.com>
	 <895334.52215.qm@web32203.mail.mud.yahoo.com>
Date: Mon, 16 Nov 2009 11:25:48 -0800
Message-ID: <1eabbac30911161125y32715fceub0d52fe98e68dd7e@mail.gmail.com>
Subject: Re: Map-reduce sorting on multiple keys
From: Something Something <mailinglists19@gmail.com>
To: general@hadoop.apache.org
Content-Type: multipart/alternative; boundary=0016e6d7e744148457047881fae3

--0016e6d7e744148457047881fae3
Content-Type: text/plain; charset=ISO-8859-1

Goutham,

Can you please take a look at my email titled.. "Custom Writable not
working"?  (I just sent it a few minutes ago.)  It's similar to this one.
The only difference is, instead of IntWritable I am using Text.  But the
sort in Map is not working as expected.  Can you tell me why?  Thanks.


On Mon, Nov 16, 2009 at 11:16 AM, Rajiv Maheshwari <rajivm01@yahoo.com>wrote:

> Thanks, appreciate it.
>
> Rajiv
>
> --- On Mon, 11/16/09, goutham patnaik <goutham.patnaik@gmail.com> wrote:
>
> From: goutham patnaik <goutham.patnaik@gmail.com>
> Subject: Re: Map-reduce sorting on multiple keys
> To: general@hadoop.apache.org
> Date: Monday, November 16, 2009, 11:08 AM
>
> Rajiv,
>
> You could write your own class which implements the WritableComparable
> interface and use this as your key class -  all u need to do is implement
> the write, readFields and compareTo methods - the map will then sort your
> keys using this method :
>
> public class TupleKey implements WritableComparable {
>  IntWritable k1;
>  IntWritable k2;
> .......
> }
>
> On Mon, Nov 16, 2009 at 9:13 AM, Rajiv Maheshwari <rajivm01@yahoo.com
> >wrote:
>
> > Hi everyone,
> >
> > I have a need to sort the output of map on 2 keys (key1, key2) - first on
> > key1, then on key2.
> >
> > Example:
> > key1  key2   values
> > -------------------------
> > 0001  0001  values...
> > 0001  0002  values...
> > 0002  0001  values...
> > 0002  0005  values...
> >
> >
> > I am thinking of the following solution approach:
> >
> > Define KEY = key1, key2   /* concatenate keys */. Override default
> > HashPartitioner and use only key1 in hashCode computation.
> >
> >
> > public class HashPartitioner<K2, V2> implements Partitioner<K2, V2> {
> >
> > public void configure(JobConf job) {}
> >
> > public int getPartition(K2 key, V2 value, int numPartitions) {
> >
> >     return (key.getKey1().hashCode() & Integer.MAX_VALUE) %
> numPartitions;
> >     }
> > }
> >
> > Would this work?
> >
> > Does anyone have any better ideas?
> >
> > Thanks much,
> > Rajiv
> >
> >
> >
> >
> >
>
>
>
>
>

--0016e6d7e744148457047881fae3--