hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohammad Tariq <donta...@gmail.com>
Subject Re: MapReduce on Local files
Date Wed, 03 Apr 2013 09:46:22 GMT
Hello Harsh,

        Thank you for the response. I am sorry for being unclear.
Actually I was talking about the backup files which end with "~"
I mean these files are not visible normally, but my job is able to
see them. Does FileInputFormat behave in the same way for "~"
as it does in the case of "." and "_"?

Thanks.

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com


On Wed, Apr 3, 2013 at 7:45 AM, Harsh J <harsh@cloudera.com> wrote:

> Not quite sure if I got your question. These tidbits may help though,
> from what I can understand:
>
> * LocalFileSystem's listing uses Java's APIs for file/dir listing, and
> has no concept of what a hidden file is on its own. It retrieves the
> whole list.
> * MR's FileInputFormat (and normal derivatives) does filter away "."
> and "_" starting path names, from added input paths to the job.
>
> On Wed, Apr 3, 2013 at 3:09 AM, Mohammad Tariq <dontariq@gmail.com> wrote:
> >
> > Warm Regards,
> > Tariq
> > https://mtariq.jux.com/
> > cloudfront.blogspot.com
> >
> >
> > ---------- Forwarded message ----------
> > From: Mohammad Tariq <dontariq@gmail.com>
> > Date: Tue, Apr 2, 2013 at 5:16 PM
> > Subject: MapReduce on Local files
> > To: mapreduce-user@hadoop.apache.org
> >
> >
> > Hello list,
> >
> >            Is a MR job capable of reading even the hidden temp files
> present
> > inside a directory located on my local FS?I have noticed this thing today
> > for the first time because till now I never tried running MR jobs on
> local
> > files.
> >
> > Thank you so much for your time?
> >
> > Warm Regards,
> > Tariq
> > https://mtariq.jux.com/
> > cloudfront.blogspot.com
> >
>
>
>
> --
> Harsh J
>

Mime
View raw message