hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ruyue Ma (JIRA)" <j...@apache.org>
Subject [jira] Created: (MAPREDUCE-1667) resource estimation still works badly in some cases
Date Fri, 02 Apr 2010 09:18:27 GMT
resource estimation still works badly in some cases
---------------------------------------------------

                 Key: MAPREDUCE-1667
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1667
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: jobtracker
            Reporter: Ruyue Ma
            Assignee: Ruyue Ma


A premise upon which our current implemention of ResourceEstimator is that the MapInputSize
and MapOutputSize has relation. 

In many user cases, the premise is not satisfied.  

e.g. 

1. The map input is a file list.
2. Every mapper will download the file, process, then output (into map intermediate output)

if one file name is very long, the estimated output size maybe very big.



-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message