hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Too many fetch-failures
Date Mon, 03 Dec 2012 21:20:45 GMT
What version/distribution of Hadoop is this?

On Mon, Dec 3, 2012 at 10:51 PM, Ajay Srivastava
<Ajay.Srivastava@guavus.com> wrote:
> Hi,
>
> I am getting following errors -
>
> 12/12/03 17:11:42 INFO mapred.JobClient: Task Id : attempt_201212031650_0001_m_000011_0,
Status : FAILED
> Too many fetch-failures
> attempt_201212031650_0001_m_000011_0: WARNING: org.apache.hadoop.metrics.jvm.EventCounter
is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties
files.
> 12/12/03 17:11:47 INFO mapred.JobClient:  map 99% reduce 24%
> 12/12/03 17:12:04 INFO mapred.JobClient:  map 99% reduce 32%
> 12/12/03 17:12:07 INFO mapred.JobClient:  map 99% reduce 33%
> 12/12/03 17:12:13 INFO mapred.JobClient:  map 99% reduce 34%
> 12/12/03 17:12:22 INFO mapred.JobClient:  map 99% reduce 35%
> 12/12/03 17:12:28 INFO mapred.JobClient:  map 99% reduce 36%
> 12/12/03 17:12:29 INFO mapred.JobClient:  map 100% reduce 36%
> 12/12/03 17:12:39 INFO mapred.JobClient:  map 100% reduce 37%
> 12/12/03 17:12:46 INFO mapred.JobClient:  map 100% reduce 38%
> 12/12/03 17:12:55 INFO mapred.JobClient:  map 100% reduce 39%
> 12/12/03 17:13:08 INFO mapred.JobClient:  map 100% reduce 40%
> 12/12/03 17:13:23 INFO mapred.JobClient:  map 100% reduce 41%
> 12/12/03 17:13:59 INFO mapred.JobClient: Task Id : attempt_201212031650_0001_m_000012_0,
Status : FAILED
> Too many fetch-failures
> attempt_201212031650_0001_m_000012_0: WARNING: org.apache.hadoop.metrics.jvm.EventCounter
is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties
files.
> 12/12/03 17:14:04 INFO mapred.JobClient:  map 99% reduce 41%
>
> I see following error in task tracker log :-
>
> 2012-12-03 17:09:40,969 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 192.168.156.22:50060,
dest: 192.168.156.22:42666, bytes: 131072, op: MAPRED_SHUFFLE, cliID: attempt_201212031650_0001_m_000166_0,
duration: 3343330
> 2012-12-03 17:09:40,970 ERROR org.mortbay.log: /mapOutput
> java.lang.IllegalStateException: Committed
>         at org.mortbay.jetty.Response.resetBuffer(Response.java:1023)
>         at org.mortbay.jetty.Response.sendError(Response.java:240)
>         at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3627)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>         at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
>         at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>
> What could be the reason behind it ? The job should have finished in 10 min. but it's
going on and on for hours.
>
>
> Regards,
> Ajay Srivastava
>
>
>



-- 
Harsh J

Mime
View raw message