crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Ortiz <dor...@videologygroup.com>
Subject RE: Read an empty directory
Date Mon, 18 May 2015 15:34:31 GMT
For the moment, seeing if I can do it with FileSystem.exists(Path) and just omit that particular
input.  Not sure if it will like the globbed fs though.

From: Josh Wills [mailto:jwills@cloudera.com]
Sent: Monday, May 18, 2015 11:31 AM
To: user@crunch.apache.org
Subject: Re: Read an empty directory

Not a really clean way; one option would be to call getSize(Configuration) on the underlying
Source you're reading from, and if it returns zero, substitute in a Pipeline.emptyPCollection()
in the place where you would have called pipeline.read(Source).

J

On Mon, May 18, 2015 at 6:09 AM, David Ortiz <dpo5003@gmail.com<mailto:dpo5003@gmail.com>>
wrote:
Hello,

     Is there a way to get my pipeline to load an empty collection instead of throwing an
exception when I pass in a globbed path that doesn't match anything?  It's part of a list
of paths that otherwise match things.

Thanks,
     Dave



--
Director of Data Science
Cloudera<http://www.cloudera.com>
Twitter: @josh_wills<http://twitter.com/josh_wills>
This email is intended only for the use of the individual(s) to whom it is addressed. If you
have received this communication in error, please immediately notify the sender and delete
the original email.
Mime
View raw message