hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Cen <cenyo...@gmail.com>
Subject Re: How to use JobConf.setKeyFieldPartitionerOptions() method
Date Tue, 24 Feb 2009 02:27:20 GMT
Thanks Jason, It works.

2009/2/23 jason hadoop <jason.hadoop@gmail.com>

> For reasons that are not clear, in 19, the partitioner steps one character
> past the end of the field unless you are very explicit in your key
> specification.
> One would assume that -k2 would pick up the second token, even if it was
> the
> last field in the key, but -k2,2 is required
>
> As near as I can tell the -kX syntax means piece X including the separator
> character, which will of course not be present, if this is the last piece.
>
> In your case, try *setKeyFieldPartitionerOptions("-k 1,1");* I believe it
> will work.
>
>
> On Sun, Feb 22, 2009 at 12:55 AM, Nick Cen <cenyongh@gmail.com> wrote:
>
> > Hi All,
> >
> > Assume the output key from the mapper has the format "k1,k2", what i
> wanna
> > to do is to use the k1 instead the whole key to partition the output,
> what
> > parameter value shoud i provide to the setKeyFieldPartitionerOptions(). i
> > have try "-k 1", but it throuth an ArrayIndexOutOfBound Exception, Thanks
> > in
> > advance.
> >
> > --
> > http://daily.appspot.com/food/
> >
>



-- 
http://daily.appspot.com/food/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message