Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hadoop-dev@lucene.apache.org
Message-ID: <8747136.1170799241851.JavaMail.jira@brutus>
Date: Tue, 6 Feb 2007 14:00:41 -0800 (PST)
From: "Owen O'Malley (JIRA)" <jira@apache.org>
To: hadoop-dev@lucene.apache.org
Subject: [jira] Updated: (HADOOP-984) Exception while retrieving a map
 output index file is not reported to JobTracker
In-Reply-To: <12906181.1170790927531.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HADOOP-984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Owen O'Malley updated HADOOP-984:
---------------------------------

    Fix Version/s: 0.11.1
         Assignee: Arun C Murthy

> Exception while retrieving a map output index file is not reported to JobTracker
> --------------------------------------------------------------------------------
>
>                 Key: HADOOP-984
>                 URL: https://issues.apache.org/jira/browse/HADOOP-984
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.11.0
>            Reporter: Nigel Daley
>         Assigned To: Arun C Murthy
>            Priority: Critical
>             Fix For: 0.11.1
>
>
> The index file of a map output reported a checksum error and was moved to "bad_file" dir.  Reduces, however, continued to attempt retrieval of the file forever.  It seem the checksum exception did *NOT* result in the map's TaskTracker reporting to the JobTracker that the map output was lost.  This was likely introduced by HADOOP-331.
> TaskTracker log:
> 2007-02-06 09:06:19,200 WARN org.apache.hadoop.dfs.DistributedFileSystem: Moving bad file /foo/testbase/tmp/mapred/local2/task_0002_m_011745_0/file.out.index to /foo/bad_files/file.out.index.-48633003
> 2007-02-06 09:06:19,203 INFO org.apache.hadoop.fs.DataInputStream: Found checksum error: org.apache.hadoop.fs.ChecksumException: Checksum error: /foo/testbase/tmp/mapred/local2/task_0002_m_011745_0/file.out.index at 363792
>         at org.apache.hadoop.fs.FSDataInputStream$Checker.verifySum(FSDataInputStream.java:167)
>         at org.apache.hadoop.fs.FSDataInputStream$Checker.read(FSDataInputStream.java:125)
>         at org.apache.hadoop.fs.FSDataInputStream$PositionCache.read(FSDataInputStream.java:218)
>         at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
>         at java.io.BufferedInputStream.read(BufferedInputStream.java:235)
>         at org.apache.hadoop.fs.FSDataInputStream$Buffer.read(FSDataInputStream.java:281)
>         at org.apache.hadoop.fs.FSDataInputStream$Buffer.seek(FSDataInputStream.java:268)
>         at org.apache.hadoop.fs.FSDataInputStream.seek(FSDataInputStream.java:331)
>         at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:1533)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
>         at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:427)
>         at org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHandler.java:475)
>         at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:567)
>         at org.mortbay.http.HttpContext.handle(HttpContext.java:1565)
>         at org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationContext.java:635)
>         at org.mortbay.http.HttpContext.handle(HttpContext.java:1517)
>         at org.mortbay.http.HttpServer.service(HttpServer.java:954)
>         at org.mortbay.http.HttpConnection.service(HttpConnection.java:814)
>         at org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:981)
>         at org.mortbay.http.HttpConnection.handle(HttpConnection.java:831)
>         at org.mortbay.http.SocketListener.handleConnection(SocketListener.java:244)
>         at org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:357)
>         at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:534)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.