crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Wills <>
Subject Re: Plan Dotfile in Configuration
Date Wed, 01 Jul 2015 21:02:57 GMT

I'm not sure what the intent was there-- Christian?


On Wed, Jul 1, 2015 at 12:29 PM, Bryan Baugher <> wrote:

> We recently ran into an issue where our code to serialize a pipeline's
> configuration was throwing an exception because one of the key/values in
> the config was too big (65k characters). We found this key/value was
> 'crunch.planner.dotfile' which is included in the pipelines config from
> Crunch.
> My question is why does Crunch provide this value into the config object?
> Crunch saves the dotfile string in the MRExecutor context[1] and I don't
> think any pipeline would need this at runtime. It also seems like there are
> no references to this config value anywhere within Crunch other then to
> write the value into the config object.
> [1] -

View raw message