hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lincoln Ritter" <linc...@lincolnritter.com>
Subject Re: Determining "Hidden" Files in Hadoop
Date Fri, 27 Jun 2008 17:22:59 GMT
Thanks.

I can see from the  private hiddenFileFilter (used by listPaths) that
'.' and '_' prefixed stuff is considered hidden, I just want to make
sure that this is "standard".

I'm working on getting Nutch 0.9 working with Hadoop 0.17 and hidden
files ("_logs") have been causing some issues.  Granted, you can
configure around this, but I've been looking for other solutions as
well.

If the hidden file behavior is well defined, it would be nice to
provide documentation, and a public interface for determining file
visibility.  Seems to me that splitting off 'hiddenFileFilter' into
its own class or providing an accessor would be sufficient.

-lincoln

--
lincolnritter.com



On Fri, Jun 27, 2008 at 10:11 AM, Doug Cutting <cutting@apache.org> wrote:
> Lincoln Ritter wrote:
>>
>> Is there a defined standard for hidden files or a public interface for
>> determining file visibility?
>
> MapReduce's FileInputFormat, and its many subclasses, ignore files and
> directories whose names begin with either "." or "_".  However FsShell's
> 'ls' and 'lsr' commands do not currently hide any files, nor do any other
> parts of Hadoop, so far as I can recall.
>
> Doug
>

Mime
View raw message