hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Félix López <jaaaelpum...@gmail.com>
Subject Re: Folder as Input and update the folder while the task is running
Date Thu, 28 Jun 2012 06:36:39 GMT

2012/6/28 Harsh J <harsh@cloudera.com>

> The job starts up with a static list of paths and splits derived from
> it during submission time. Input files added to the directory after
> the job has been submitted are not considered.
> On Wed, Jun 27, 2012 at 11:15 PM, Félix López <jaaaelpumuki@gmail.com>
> wrote:
> > I have a task with a folder as input: FileInputFormat.setInputPaths(job,
> new
> > Path("/folder"));
> >
> > What happens when the task is running and I write new files in the
> folder?
> > The task receive the new files or not?
> >
> > Thanks
> --
> Harsh J


It's easier to ask forgiveness than it is to get permission

".....it doesn't matter how many times you fail. It doesn't matter how many
times you almost get it right. No one is going to know or care about your
failures, and either should you. All you have to do is learn from them and
those around you because...All that matters in business is that you get it
right once. Then everyone can tell you how lucky you are."
--Mark Cuban"

"Always be the worst guy in every band you're in." If you're the best guy
there, you need to be in a different band. And I think that works for
almost everything that's out there as well." Pat Metheny

View raw message