hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: how to skip a mapper
Date Mon, 10 Sep 2012 11:38:03 GMT

Yes this is possible (and actually does happen in regular MR scenario
anyway - when the input is split across several locations). You'll
need a custom InputFormat#getSplits implementation to do this (create
input splits with the first offset itself set to the known offset
location, instead of 0).

On Mon, Sep 10, 2012 at 5:01 PM, Anit Alexander <anitamalex@gmail.com> wrote:
> Hello list,
>       Is it possible to start the mapper from a particular byte
> location in a file which is in hdfs?
> Regards,
> Anit

Harsh J

View raw message