hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] Created: (MAPREDUCE-2389) Spurious EOFExceptions reading SpillRecord index files
Date Wed, 16 Mar 2011 06:51:29 GMT
Spurious EOFExceptions reading SpillRecord index files
------------------------------------------------------

                 Key: MAPREDUCE-2389
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2389
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: tasktracker
    Affects Versions: 0.22.0
         Environment: Seen on RHEL 5.5, RHEL 6.0, local dirs on ext3, Java 6u20 and 6u24
            Reporter: Todd Lipcon
            Priority: Critical


In large jobs, I see around 1 shuffle fetch out of every million fetches fail with an EOFException
reading the SpillRecord index file. After lots of investigation, including systemtap, it looks
like the read() syscall is actually returning a premature "0" result for no reason, so this
is likely a kernel or filesystem bug which is exacerbated by some workload the TT does.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message