incubator-crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Wills <josh.wi...@gmail.com>
Subject Re: Exceptions throw in code
Date Tue, 07 Aug 2012 16:07:58 GMT
Hey Jonathan,

Try calling Pipeline.enableDebug before you kick off your job-- I
think it does what you're looking for, at the expense of some
additional overhead for extra logging and try-catch blocks.

J

On Tue, Aug 7, 2012 at 9:02 AM, Jonathan Dinu <jondinu@gmail.com> wrote:
> Hello,
>
> I am curious as to how exceptions get handled inside the Crunch runtime.  I am trying
to do some input/output error checking and throwing the appropriate exceptions.  The exceptions
in my code are getting throw but seem to be eaten by Crunch and instead of my Exception (throw
in a MapFn) bubbling up to the console, a Crunch runtime exception is throw instead:
>
> org.apache.crunch.impl.mr.run.CrunchRuntimeException: java.io.IOException: No files found
to materialize at: /tmp/crunch72870282/p3
>
> Is there anyway to break out of a parallelDo or a way gracefully handle things such as
bad data formats encountered?  Or do I need to just catch my exceptions and give Crunch an
appropriate response.
>
> I tried throwing a RuntimeException in my run() method as well as the MapFn's map() but
both of those do not seem to stop Crunch from trying to materialize the collection and subsequently
throw a Crunch RuntimeException.  Any help is greatly appreciated.
>
> thanks!

Mime
View raw message