hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Enis Soztutar (JIRA)" <j...@apache.org>
Subject [jira] Resolved: (HADOOP-1305) extra space in mapred.local.dir config results in FileNotFoundException in reduce copyOutput
Date Thu, 23 Aug 2007 12:00:42 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-1305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Enis Soztutar resolved HADOOP-1305.
-----------------------------------

    Resolution: Duplicate

This, and similar issues, will probably be solved in HADOOP-785.

> extra space in mapred.local.dir config results in FileNotFoundException in reduce copyOutput
> --------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-1305
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1305
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Koji Noguchi
>            Priority: Minor
>
> When mis-configured with extra space in mapred.local.dir,  TaskTracker comes up and run
tasks but always fail when reduce tries to pull the map output.
> It would be nice if this error is caught earlier.
> example (in hadoop-.0.12.3)
> <value>[space]/hadoop/mapred/local</value>
> will result in 
> at the mapper side.
> 2007-04-30 16:35:23,260 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(task_0001_m_000000_0,0)
failed :
> java.io.FileNotFoundException: /hadoop/mapred/local/task_0001_m_000000_0/file.out.index
>   at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:332)
>   at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:245)
>   at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:1637)
>   at javax.servlet.http.HttpServlet.service(HttpServlet.java:689)
>   at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
>   at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:427)
>   at org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicationHandler.java:475)
>   at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:567)
>   at org.mortbay.http.HttpContext.handle(HttpContext.java:1565)
>   at org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplicationContext.java:635)
>   at org.mortbay.http.HttpContext.handle(HttpContext.java:1517)
>   at org.mortbay.http.HttpServer.service(HttpServer.java:954)
>   at org.mortbay.http.HttpConnection.service(HttpConnection.java:814)
>   at org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:981)
>   at org.mortbay.http.HttpConnection.handle(HttpConnection.java:831)
>   at org.mortbay.http.SocketListener.handleConnection(SocketListener.java:244)
>   at org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:357)
>   at org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:534)
> at the reducer side.
> 2007-04-30 16:35:23,271 WARN org.apache.hadoop.mapred.TaskRunner: java.io.FileNotFoundException:
http://aaa.bbb.com:11111/mapOutput?map=task_0001_m_000000_0&reduce=0
>   at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1243)
>   at org.apache.hadoop.mapred.MapOutputLocation.getFile(MapOutputLocation.java:200)
>   at org.apache.hadoop.mapred.ReduceTaskRunner$MapOutputCopier.copyOutput(ReduceTaskRunner.java:311)
>   at org.apache.hadoop.mapred.ReduceTaskRunner$MapOutputCopier.run(ReduceTaskRunner.java:274)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message