hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kihwal Lee (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3851) Allow more aggressive action on detection of the jetty issue
Date Mon, 13 Feb 2012 22:43:04 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207297#comment-13207297
] 

Kihwal Lee commented on MAPREDUCE-3851:
---------------------------------------

@Todd If anyone can provide evidence or even a stat showing that the jetty exception behavior
converges to spinning selector thread behavior in a short amount of time, I see no reason
to have this jira.  

I've seen cases where some trackers having quite a few of the exceptions reported in MAPREDUCE-2529
while many new requests were served successfully. The fault-detection coverage of 100% is
very critical to this problem. If it's not 100%, jobs with a large number of mappers will
still suffer. Other than the potential overlap of coverage, do you see any risk in this? 
We want the threshold to be configurable and the default can be either to 0 (disable) or very
high.
                
> Allow more aggressive action on detection of the jetty issue
> ------------------------------------------------------------
>
>                 Key: MAPREDUCE-3851
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3851
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: tasktracker
>    Affects Versions: 1.0.0
>            Reporter: Kihwal Lee
>             Fix For: 1.1.0, 1.0.1
>
>
> MAPREDUCE-2529 added the useful failure detection mechanism. In this jira, I propose
we add a periodic check inside TT and configurable action to self-destruct. Blacklisting helps
but is not enough. Hung jetty still accepts connection and it takes very long time for clients
to fail out. Short jobs are delayed for hours because of this. This feature will be a nice
companion to MAPREDUCE-3184.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message