hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: Determining "Hidden" Files in Hadoop
Date Fri, 27 Jun 2008 18:37:46 GMT
Lincoln Ritter wrote:
> I can see from the  private hiddenFileFilter (used by listPaths) that
> '.' and '_' prefixed stuff is considered hidden, I just want to make
> sure that this is "standard".

Yes, it is standard for mapreduce input and output directories.

> I'm working on getting Nutch 0.9 working with Hadoop 0.17 and hidden
> files ("_logs") have been causing some issues.  Granted, you can
> configure around this, but I've been looking for other solutions as
> well.
> If the hidden file behavior is well defined, it would be nice to
> provide documentation, and a public interface for determining file
> visibility.  Seems to me that splitting off 'hiddenFileFilter' into
> its own class or providing an accessor would be sufficient.

If Nutch cannot extend FileInputFormat then, yes, we should make this 
filter public.  If that's the case, please submit a patch.



View raw message