hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (HADOOP-2771) changing the number of reduces dramatically changes the time of the map time
Date Fri, 01 Aug 2008 22:49:31 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12619154#action_12619154
] 

chris.douglas edited comment on HADOOP-2771 at 8/1/08 3:49 PM:
---------------------------------------------------------------

Does this job run with a combiner? That would also add time spent in each spill in proportion
to the number of reducers. Is there large variation in the map execution time, or are most
fairly close to the average time?

      was (Author: chris.douglas):
    Does this job run with a combiner? That would also add time spent in each spill in proportion
to the number of reducers.
  
> changing the number of reduces dramatically changes the time of the map time
> ----------------------------------------------------------------------------
>
>                 Key: HADOOP-2771
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2771
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.17.1
>            Reporter: Owen O'Malley
>
> By changing the number of reduces, the time for an individual map changes radically.
By running the same program and data with different numbers of reduces (2500, 7500, 25000)
the times for each map changed radically (0:50, 1:20, 5h).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message