hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christian Kunz (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-2771) changing the number of reduces dramatically changes the time of the map time
Date Thu, 24 Jul 2008 01:56:31 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12616327#action_12616327
] 

Christian Kunz commented on HADOOP-2771:
----------------------------------------

Still an issue in 0.17.1:

Sample job with 100,000 maps:

9,000  reduces: map executes on average in 1hr
18,000 reduces: map executes on average in 1:40 hr

> changing the number of reduces dramatically changes the time of the map time
> ----------------------------------------------------------------------------
>
>                 Key: HADOOP-2771
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2771
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.17.1
>            Reporter: Owen O'Malley
>
> By changing the number of reduces, the time for an individual map changes radically.
By running the same program and data with different numbers of reduces (2500, 7500, 25000)
the times for each map changed radically (0:50, 1:20, 5h).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message