hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: MapReduce on Local files
Date Wed, 03 Apr 2013 02:15:04 GMT
Not quite sure if I got your question. These tidbits may help though,
from what I can understand:

* LocalFileSystem's listing uses Java's APIs for file/dir listing, and
has no concept of what a hidden file is on its own. It retrieves the
whole list.
* MR's FileInputFormat (and normal derivatives) does filter away "."
and "_" starting path names, from added input paths to the job.

On Wed, Apr 3, 2013 at 3:09 AM, Mohammad Tariq <dontariq@gmail.com> wrote:
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> ---------- Forwarded message ----------
> From: Mohammad Tariq <dontariq@gmail.com>
> Date: Tue, Apr 2, 2013 at 5:16 PM
> Subject: MapReduce on Local files
> To: mapreduce-user@hadoop.apache.org
>
>
> Hello list,
>
>            Is a MR job capable of reading even the hidden temp files present
> inside a directory located on my local FS?I have noticed this thing today
> for the first time because till now I never tried running MR jobs on local
> files.
>
> Thank you so much for your time?
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>



-- 
Harsh J

Mime
View raw message