hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amandeep Khurana <ama...@gmail.com>
Subject Re: Hadoop Input Files Directory
Date Fri, 11 Sep 2009 21:33:22 GMT
You can give something like /path/to/directories/*/*/*


On Fri, Sep 11, 2009 at 2:10 PM, Boyu Zhang <bzhang@cs.utsa.edu> wrote:

> Dear All,
>
>
>
> I have an input directories of depth 3, the actual files are in the deepest
> levels. (something like /data/user/dir_0/file0 , /data/user/dir_1/file0,
> /data/user/dir_2/file0) And I want to write a mapreduce job to process
> these
> files in the deepest levels.
>
>
>
> One way of doing so is to specify the input path to the directories that
> contain the files, like /data/user/dir_0, /data/user/dir_1,
> /data/user/dir_2. But this way is not feasible when I have much more
> directories as I will. I tried to specify the input path as /data/user, but
> I get error of cannot open filename /data/user/dir_0.
>
>
>
> My question is that is there any way that I can process all the files in a
> hierarchy with the input path set to the top level?
>
>
>
> Thanks a lot for the time!
>
>
>
> Boyu Zhang
>
> University of Delaware
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message