crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Wills <jwi...@cloudera.com>
Subject Re: More plan details
Date Mon, 18 May 2015 15:31:45 GMT
+1, that would be good to have. The Spotify folks also requested an
enhancement to print out a recommended number of reducers for the *next*
run of a job based on the performance of the last run, which I think should
be do-able w/o too much trouble.

J

On Mon, May 18, 2015 at 6:42 AM, Micah Whitacre <mkwhitacre@gmail.com>
wrote:

> It doesn't look like the current DOT file writer includes functionality to
> write out that information.
>
> Feel free to log an enhancement to get that functionality.
> https://issues.apache.org/jira/browse/CRUNCH
>
> On Mon, May 18, 2015 at 8:32 AM, Ron Hashimshony <
> ron.hashimshony@myheritage.com> wrote:
>
>> Hi,
>> It seems that the planning dot is lacking the planned number of reducer,
>> which is computed from the input files size.
>> Is there any way of adding both the speculated file sizes that crunch is
>> defining for each job, and the number of reducers it is going to run?
>> Thanks,
>> Ron.
>>
>
>


-- 
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>

Mime
View raw message