hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Error reading task output and (log)FileNotFoundExceptions
Date Wed, 04 Jan 2012 01:49:26 GMT
Markus,

What's the permissions on /opt/hadoop/hadoop-0.20.205.0/logs (-la)? Does this happen only
on a single node?

On 04-Jan-2012, at 3:25 AM, Markus Jelsma wrote:

> Hi,
> 
> On our 0.20.205.0 test cluster we sometimes see tasks failing for no clear 
> reason. The task tracker logs show us:
> 
> 2012-01-03 21:16:27,256 WARN org.apache.hadoop.mapred.TaskLog: Failed to 
> retrieve stdout log for task: attempt_201201031651_0008_m_000233_0
> java.io.FileNotFoundException: 
> /opt/hadoop/hadoop-0.20.205.0/libexec/../logs/userlogs/job_201201031651_0008/attempt_201201031651_0008_m_000233_0/log.index

> (No such file or directory)
>        at java.io.FileInputStream.open(Native Method)
>        at java.io.FileInputStream.<init>(FileInputStream.java:120)
>        at 
> org.apache.hadoop.io.SecureIOUtils.openForRead(SecureIOUtils.java:102)
>        at 
> org.apache.hadoop.mapred.TaskLog.getAllLogsFileDetails(TaskLog.java:187)
>        at org.apache.hadoop.mapred.TaskLog$Reader.<init>(TaskLog.java:422)
>        at 
> org.apache.hadoop.mapred.TaskLogServlet.printTaskLog(TaskLogServlet.java:81)
>        at 
> org.apache.hadoop.mapred.TaskLogServlet.doGet(TaskLogServlet.java:296)
>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>        at 
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>        at 
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
>        at 
> org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:835)
>        at 
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>        at 
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
>        at 
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>        at 
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>        at 
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
>        at 
> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>        at 
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>        at 
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>        at org.mortbay.jetty.Server.handle(Server.java:326)
>        at 
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>        at 
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> 
> However, if we inspect the log more closely we actually see it happening 
> several times but only one seems to be thrown simultaneously with the task 
> failing. We see no errors in the datanode's log.
> 
> I've been looking through the configuration descriptions of 0.20.205.0 but 
> didn't find a setting that could be fix this or is responsible for this.
> 
> Any hints? Upgrade to 1.0? Patches?
> Thanks


Mime
View raw message