incubator-crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gabriel Reid <gabriel.r...@gmail.com>
Subject Re: MemPipeline and context
Date Wed, 30 Jan 2013 19:51:59 GMT
Hi Tim,

On Wed, Jan 30, 2013 at 10:33 AM, Tim van Heugten <stimme@gmail.com> wrote:

>
> Since april I'm using Crunch for a project. We're not doing only linear
> executions of the pipeline, so we're sometimes having issues with how
> Crunch is optimizing our execution graph. We need to add materializations
> here and there as hints to what parts of the graph can be shared for
> outputs and so on.
>

About the extra calls to materialize to force changes to the execution
plan: I remember seeing this previously. We've discussed adding something
specifically for this functionality to the API, although it hasn't yet
happened.

Could you give an example of a situation where these extra materialize
calls get added? That would be useful for validating the addition to the
API.


Thanks,

Gabriel

Mime
View raw message