hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3140) JobTracker should not try to promote a (map) task if it does not write to DFS at all
Date Thu, 03 Apr 2008 20:23:28 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12585273#action_12585273

Devaraj Das commented on HADOOP-3140:

Dhruba, is that a documented exception. I didn't see it in the FileSystem.getContentSummary
API doc. So if it is not documented is it advisable to bank client code on the exception?
For e.g., what if getContentSummary, later on, returns null for non existent paths? So, unless
FileSystem provides a guarantee that an exception will be thrown for non-existent paths, i'd
like to go in the lines of what Amar mentioned in the code snippet. Thoughts?

> JobTracker should not try to promote a (map) task if it does not write to DFS at all
> ------------------------------------------------------------------------------------
>                 Key: HADOOP-3140
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3140
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Runping Qi
>            Assignee: Amar Kamat
>             Fix For: 0.17.0
>         Attachments: HADOOP-3140-v1.patch, HADOOP-3140-v2.patch
> In most cases, map tasks do not write to dfs.
> Thus, when they complete, they should not be put into commit_pending queue at all.
> This will improve the task promotion significantly.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message