hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3807) task attempt failing to report status just after the intialization
Date Thu, 14 Aug 2008 10:47:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622511#action_12622511
] 

Amar Kamat commented on HADOOP-3807:
------------------------------------

Even I am seeing this error in the test case I have for HADOOP-3245. 
{noformat}
[junit] 2008-08-14 15:43:07,296 ERROR mapred.TaskTracker (TaskTracker.java:offerService(1039))
- Caught exception: java.io.IOException: Call to localhost/127.0.0.1:34192 failed on local
exception: null
   [junit]     at org.apache.hadoop.ipc.Client.call(Client.java:710)
   [junit]     at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:216)
   [junit]     at org.apache.hadoop.mapred.$Proxy7.heartbeat(Unknown Source)
   [junit]     at org.apache.hadoop.mapred.TaskTracker.transmitHeartBeat(TaskTracker.java:1105)
   [junit]     at org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:942)
   [junit]     at org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:1448)
   [junit]     at org.apache.hadoop.mapred.MiniMRCluster$TaskTrackerRunner.run(MiniMRCluster.java:186)
   [junit]     at java.lang.Thread.run(Thread.java:619)
   [junit] Caused by: java.io.EOFException
   [junit]     at java.io.DataInputStream.readInt(DataInputStream.java:375)
   [junit]     at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:499)
   [junit]     at org.apache.hadoop.ipc.Client$Connection.run(Client.java:441)
   [junit]
   [junit] 2008-08-14 15:43:07,296 INFO  mapred.TaskTracker (TaskTracker.java:transmitHeartBeat(1081))
- Resending 'status' to 'localhost' with reponseId '0
   [junit] 2008-08-14 15:43:07,297 INFO  mapred.JobTracker (JobTracker.java:heartbeat(1637))
- Got heartbeat from: tracker_host.com:localhost/127.0.0.1:33437 (initialContact: false acceptNewTasks:
true) with responseId: 0
   [junit] 2008-08-14 15:43:07,298 INFO  mapred.JobTracker (JobTracker.java:addTaskTrackerStatus(1509))
- Adding tracker name tracker_host.com:localhost/127.0.0.1:33437
   [junit] 2008-08-14 15:43:07,298 WARN  mapred.JobTracker (JobTracker.java:processHeartbeat(1873))
- Status from unknown Tracker : tracker_host.com:localhost/127.0.0.1:33437
   [junit] 2008-08-14 15:43:07,301 INFO  mapred.TaskTracker (TaskTracker.java:reinitTaskTracker(1156))
- Recieved RenitTrackerAction from JobTracker
   [junit] 2008-08-14 15:43:07,303 WARN  mapred.TaskTracker (TaskTracker.java:run(1469)) -
Reinitializing local state
   [junit] 2008-08-14 15:43:07,303 INFO  jvm.JvmMetrics (JvmMetrics.java:init(62)) - Cannot
initialize JVM Metrics with processName=TaskTracker, sessionId= - already initialized
   [junit] 2008-08-14 15:43:07,304 INFO  mapred.TaskTracker (TaskTracker.java:run(503)) -
Shutting down: Map-events fetcher for all reduce tasks on tracker_host.com:localhost/127.0.0.1:33437
   [junit] 2008-08-14 15:43:07,304 INFO  ipc.Server (Server.java:stop(992)) - Stopping server
on 33437
   [junit] 2008-08-14 15:43:07,305 INFO  metrics.RpcMetrics (RpcMetrics.java:<init>(56))
- Initializing RPC Metrics with hostName=TaskTracker, port=49696
   [junit] 2008-08-14 15:43:07,305 INFO  ipc.Server (Server.java:run(330)) - Stopping IPC
Server listener on 33437
   [junit] 2008-08-14 15:43:07,305 INFO  ipc.Server (Server.java:run(448)) - IPC Server Responder:
starting
   [junit] 2008-08-14 15:43:07,308 INFO  ipc.Server (Server.java:run(291)) - IPC Server listener
on 49696: starting
   [junit] 2008-08-14 15:43:07,309 INFO  ipc.Server (Server.java:run(920)) - IPC Server handler
0 on 33437: exiting
   [junit] 2008-08-14 15:43:07,309 INFO  ipc.Server (Server.java:run(920)) - IPC Server handler
1 on 33437: exiting
   [junit] 2008-08-14 15:43:07,314 INFO  ipc.Server (Server.java:run(869)) - IPC Server handler
0 on 49696: starting
   [junit] 2008-08-14 15:43:07,327 ERROR mapred.MiniMRCluster (MiniMRCluster.java:run(191))
- task tracker 0 crashed
   [junit] java.lang.NullPointerException
   [junit]     at org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java:405)
   [junit]     at org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:1470)
   [junit]     at org.apache.hadoop.mapred.MiniMRCluster$TaskTrackerRunner.run(MiniMRCluster.java:186)
   [junit]     at java.lang.Thread.run(Thread.java:619)
   [junit] 2008-08-14 15:43:07,329 INFO  ipc.Server (Server.java:run(869)) - IPC Server handler
1 on 49696: starting
   [junit] 2008-08-14 15:43:07,329 INFO  ipc.Server (Server.java:run(502)) - Stopping IPC
Server Responder
{noformat} 

The lines in {{TaskTracker.java}} that correspond to it are 
{code}
this.taskReportServer =
        RPC.getServer(this, bindAddress, tmpPort, max, false, this.fConf);
this.taskReportServer.start();
{code}

> task attempt failing to report status just after the intialization
> ------------------------------------------------------------------
>
>                 Key: HADOOP-3807
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3807
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Amareshwari Sriramadasu
>             Fix For: 0.19.0
>
>
> In sort500 runs, I noticed task attempts failing to report status : saying
> Task attempt_200807220707_0002_r_000336_1 failed to report status for 605 seconds. Killing!
> And the task logs has NullPointerException saying:
> *stderr logs*
> Exception in thread "main" java.lang.NullPointerException
> 	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2146)
> Task tracker logs for the same attempt are:
> 2008-07-22 08:07:37,110 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction:
attempt_200807220707_0002_r_000336_1
> 2008-07-22 08:17:42,144 INFO org.apache.hadoop.mapred.TaskTracker: attempt_200807220707_0002_r_000336_1:
Task attempt_200807220707_0002_r_000336_1 failed toreport status for 605 seconds. Killing!
> 2008-07-22 08:17:42,162 INFO org.apache.hadoop.mapred.TaskTracker: About to purge task:
attempt_200807220707_0002_r_000336_1
> 2008-07-22 08:17:42,163 INFO org.apache.hadoop.mapred.TaskRunner: attempt_200807220707_0002_r_000336_1
done; removing files.
> 2008-07-22 08:18:15,481 WARN org.apache.hadoop.mapred.TaskTracker: Unknown child task
finshed: attempt_200807220707_0002_r_000336_1. Ignored.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message