hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sriramadasu (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1635) ResourceEstimator does not work after MAPREDUCE-842
Date Thu, 01 Apr 2010 05:57:27 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12852259#action_12852259
] 

Amareshwari Sriramadasu commented on MAPREDUCE-1635:
----------------------------------------------------

bq. MapOutputFile is still being used in TaskTracker process space, as a member of TaskRunner
class.
I raised MAPREDUCE-1662 for this.

bq. getRunState() != TaskStatus.State.SUCCEEDED check is removed in calculateOutputSize().
I guess we need this check if some other caller also calls this method
This check is not needed because calaculateOutputSize is called from done(). If any other
caller calls it, it will return -1 because the file is not there. We cannot add the check,
since SUCCEEDED state is not set by Task, it is set by TaskTracker whenever the Task reports
done().

> ResourceEstimator does not work after MAPREDUCE-842
> ---------------------------------------------------
>
>                 Key: MAPREDUCE-1635
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1635
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: tasktracker
>    Affects Versions: 0.21.0
>            Reporter: Amareshwari Sriramadasu
>            Assignee: Amareshwari Sriramadasu
>             Fix For: 0.22.0
>
>         Attachments: patch-1635.txt
>
>
> MAPREDUCE-842 changed Child's mapred.local.dir to have attemptDir as the base local directory.
Also assumption is that
> org.apache.hadoop.mapred.MapOutputFile always gets Child's mapred.local.dir. 
> But, MapOuptutFile.getOutputFile() is called from TaskTracker's conf, which does not
find the output file. Thus TaskTracker.tryToGetOutputSize() always returns -1.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message