hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jason hadoop <jason.had...@gmail.com>
Subject Re: How to use JobConf.setKeyFieldPartitionerOptions() method
Date Mon, 23 Feb 2009 06:43:36 GMT
For reasons that are not clear, in 19, the partitioner steps one character
past the end of the field unless you are very explicit in your key
specification.
One would assume that -k2 would pick up the second token, even if it was the
last field in the key, but -k2,2 is required

As near as I can tell the -kX syntax means piece X including the separator
character, which will of course not be present, if this is the last piece.

In your case, try *setKeyFieldPartitionerOptions("-k 1,1");* I believe it
will work.


On Sun, Feb 22, 2009 at 12:55 AM, Nick Cen <cenyongh@gmail.com> wrote:

> Hi All,
>
> Assume the output key from the mapper has the format "k1,k2", what i wanna
> to do is to use the k1 instead the whole key to partition the output, what
> parameter value shoud i provide to the setKeyFieldPartitionerOptions(). i
> have try "-k 1", but it throuth an ArrayIndexOutOfBound Exception, Thanks
> in
> advance.
>
> --
> http://daily.appspot.com/food/
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message