hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ray Young (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-5) Shuffle's getMapOutput() fails with EofException, followed by IllegalStateException
Date Tue, 27 Apr 2010 02:27:34 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861209#action_12861209
] 

Ray Young commented on MAPREDUCE-5:
-----------------------------------

Is this issue solved? Since the filling of the memory is the always case in Sort like jobs,
is it preferable not to close the input stream and maintain it, also to avoid the unnecessary
reopen? This will not lead to job failure as reopen and re-transferring will occur, but the
hosts will be regarded as "slow hosts" and put into the penaltyBox, thus slowing down the
whole progress.

> Shuffle's getMapOutput() fails with EofException, followed by IllegalStateException
> -----------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 0.20.2
>         Environment: Sun Java 1.6.0_13, OpenSolaris, running on a SunFire 4150 (x64)
10 node cluster
>            Reporter: George Porter
>
> During the shuffle phase, I'm seeing a large sequence of the following actions:
> 1) WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_200905181452_0002_m_000010_0,0)
failed : org.mortbay.jetty.EofException
> 2) WARN org.mortbay.log: Committed before 410 getMapOutput(attempt_200905181452_0002_m_000010_0,0)
failed : org.mortbay.jetty.EofException
> 3) ERROR org.mortbay.log: /mapOutput java.lang.IllegalStateException: Committed
> The map phase completes with 100%, and then the reduce phase crawls along with the above
errors in each of the TaskTracker logs.  None of the tasktrackers get lost.  When I run non-data
jobs like the 'pi' test from the example jar, everything works fine.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message