hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: PathFilter File Glob
Date Mon, 27 Feb 2012 07:36:14 GMT
Hi Simon,

You need to implement your custom PathFilter derivative class, and
then set it via your {File}InputFormat class using setInputPathFilter:

http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/FileInputFormat.html#setInputPathFilter(org.apache.hadoop.mapred.JobConf,%20java.lang.Class)

(TextInputFormat is a derivative of FileInputFormat, and hence has the
same method.)

HTH.

2012/2/23 Heeg, Simon <s.heeg@telekom.de>:
> Hello,
>
> I would like to use a PathFilter for filtering the files with a regular expression which
are read by the TextInputFormat, but I don't know how to apply the filter. I cannot find a
setter. Unfortunately google was not my friend with this issue and "The definitive Guide"
does  not help that much.  I am using Hadoop 0.20.2-cdh3u3.
>

-- 
Harsh J

Mime
View raw message