hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1099) Setup and cleanup tasks could affect job latency if they are caught running on bad nodes
Date Tue, 13 Oct 2009 06:16:31 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12764968#action_12764968
] 

Hemanth Yamijala commented on MAPREDUCE-1099:
---------------------------------------------

Note that I am not suggesting we speculate setup and cleanup tasks. In an offline discussion
with Amareshwari and Devaraj we realized there are potential difficulties in speculating these
attempts. Firstly there's no good way to measure the progress of these attempts against other
attempts. I am just opening this issue to open for discussion.

> Setup and cleanup tasks could affect job latency if they are caught running on bad nodes
> ----------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1099
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1099
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker
>    Affects Versions: 0.20.1
>            Reporter: Hemanth Yamijala
>
> We found cases on our clusters where a setup task got scheduled on a bad node and took
upwards of several minutes to run, adversely affecting job runtimes. Speculation did not help
here as speculation is not used for setup tasks. I suspect the same could happen for cleanup
tasks as well.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message