crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jinal Shah <jinalshah2...@gmail.com>
Subject Re: .materialize() returns empty collection on pipeline error?
Date Wed, 28 Jan 2015 18:36:12 GMT
I think it was intented from these commits I see here
https://github.com/apache/crunch/commit/3711cea61bded4c90b235a01163ae5f855089917
and
https://github.com/apache/crunch/commit/ded504eb133fa0814e2d90ff2a662e72a67e04bb.
Josh can enhance on this more.

On Wed, Jan 28, 2015 at 9:26 AM, Mārtiņš Kalvāns <martins.kalvans@gmail.com>
wrote:

> Hi.
>
> When pipeline fails on cluster with some exception, materialize() returns
> empty collection and just logs error message.
>
> I'm (very, very) puzzled about this behaviour:
>
> https://github.com/apache/crunch/blob/master/crunch-core/src/main/java/org/apache/crunch/materialize/MaterializableIterable.java#L92
> Is this really intended behaviour?
>
> If so, then some documentation for materialize() function about this
> behaviour would be really nice to have. :)
>
>
> --
> Mārtiņš
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message