hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1374) TaskTracker falls into an infinite loop.
Date Sat, 26 May 2007 04:39:16 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12499283
] 

Devaraj Das commented on HADOOP-1374:
-------------------------------------

Regarding the calls the reduce task makes, they all look good. It would be very nice to have
a look at the tasklog of task_0001_r_000000_1 (btw from the taskid string it looks like this
is the second attempt for the same reduce TIP; the first attempt failed) since they have all
what the reduce task is doing in its life. 
Could you please attach the task logs whenever you see this happening. Also, in your  test
app do you have just one reduce? (The task logs can be found at $LOG_DIR/userlogs/<taskid>/).
Another thing worth doing is to set the value of mapred.userlog.purgesplits to false so that
we don't lose the task logs.
<property>
  <name>mapred.userlog.purgesplits</name>
  <value>false</value>
  <description>Should the splits be purged disregarding the user-log size limit.
  </description>
</property>
The task logs will hopefully give some insights.

> TaskTracker falls into an infinite loop.
> ----------------------------------------
>
>                 Key: HADOOP-1374
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1374
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.12.3
>            Reporter: Konstantin Shvachko
>         Assigned To: Arun C Murthy
>            Priority: Blocker
>             Fix For: 0.13.0
>
>         Attachments: DataNode1.log, DataNode2.log, JobTracker.log, NameNode.log, TaskTracker1.log,
TaskTracker2.log, TestDFSIO.log
>
>
> All maps had been completed successfully. I had only one reduce task during which
> TaskTracker infinitely outputs:
> 07/05/15 19:35:41 INFO mapred.TaskTracker: task_0001_r_000000_0 0.16666667% reduce >
copy (4 of 8 at 0.00 MB/s) > 
> 07/05/15 19:35:42 INFO mapred.TaskTracker: task_0001_r_000000_0 0.16666667% reduce >
copy (4 of 8 at 0.00 MB/s) > 
> 07/05/15 19:35:43 INFO mapred.TaskTracker: task_0001_r_000000_0 0.16666667% reduce >
copy (4 of 8 at 0.00 MB/s) > 
> 07/05/15 19:35:44 INFO mapred.TaskTracker: task_0001_r_000000_0 0.16666667% reduce >
copy (4 of 8 at 0.00 MB/s) > 
> 07/05/15 19:35:45 INFO mapred.TaskTracker: task_0001_r_000000_0 0.16666667% reduce >
copy (4 of 8 at 0.00 MB/s) > 
> JobTracker does not log anything about task task_0001_r_000000_0 except for
> 07/05/15 19:49:01 INFO mapred.JobTracker: Adding task 'task_0001_r_000000_0' to tip tip_0001_r_000000,
for tracker 'tracker_my-host.com:50050'

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message