hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark question <markq2...@gmail.com>
Subject Re: Can Mapper get paths of inputSplits ?
Date Fri, 13 May 2011 04:23:26 GMT
Thanks for the reply Owen, I only knew about map.input.file.

 So there is no way I can see the other possible splits (start+length)? like
some function that returns strings of map.input.file and map.input.offset of
the other mappers ?

Thanks,
Mark

On Thu, May 12, 2011 at 9:08 PM, Owen O'Malley <omalley@apache.org> wrote:

> On Thu, May 12, 2011 at 8:59 PM, Mark question <markq2011@gmail.com>
> wrote:
>
> > Hi
> >
> >   I'm using FileInputFormat which will split files logically according to
> > their sizes into splits. Can the mapper get a pointer to these splits?
> and
> > know which split it is assigned ?
> >
>
> Look at
>
> http://hadoop.apache.org/common/docs/r0.20.203.0/mapred_tutorial.html#Task+JVM+Reuse
>
>  In particular, map.input.file and map.input.offset are the configuration
> parameters that you want.
>
> -- Owen
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message