crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Wills <josh.wi...@gmail.com>
Subject Re: Removing PCollection.cache call resulting in two MR jobs writing to same path
Date Fri, 22 Jul 2016 17:27:47 GMT
Hey Ben,

That's a bit surprising; can I see a bit of the DAG?

J

On Fri, Jul 22, 2016 at 8:48 AM, Ben Juhn <benjijuhn@gmail.com> wrote:

> Hello,
>
> I removed a .cache call and am seeing some troublesome behavior.  It
> results in two nodes in Crunch's execution graph writing to the same output
> path.  When I add the .cache call back I end up with one node writing to
> crunch tmp space, and the other node writing to the output path.
>
> Is this expected behavior?
>
> Thanks,
> Ben
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message