hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Dunning" <ted.dunn...@gmail.com>
Subject Re: Getting reference to input format used by a job
Date Thu, 26 Jun 2008 21:41:04 GMT
The configure method on the mapper should be called each time a new file is

You can collect the file names there and write them to the file system in
the close method.

On Thu, Jun 26, 2008 at 1:42 PM, Nathan Marz <nathan@rapleaf.com> wrote:

> Hello,
> I have an input format which finds all files reachable from a root
> directory, and I want to delete those files found and those files only after
> the job completes. This is tricky because files are being constantly added.
> Is there a way to get a reference to the input format used so I can
> determine which files it chose? Otherwise, is there a way for the input
> format to communicate back to the "main" function this information?
> Thanks,
> Nathan Marz
> RapLeaf


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message