hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-2119) JobTracker becomes non-responsive if the task trackers finish task too fast
Date Tue, 30 Oct 2007 12:43:50 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12538760
] 

Devaraj Das commented on HADOOP-2119:
-------------------------------------

bq. Yes, the #running-mappers kept on going up and went beyond the actual number of RUNNING
mappers on the task trackers 
Ok, maybe the task commit thread was slow in processing the task completion events, and one
reason for that could be that the namenode was slow... 

bq. I've killed the jobtracker, thus cannot do "kill -3" anymore.
Since it is easy to reproduce the problem, could you please have one run and post the traces
of the jobtracker's thread stacks (kill -3, when the JT is at 100% CPU).


> JobTracker becomes non-responsive if the task trackers finish task too fast
> ---------------------------------------------------------------------------
>
>                 Key: HADOOP-2119
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2119
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Runping Qi
>             Fix For: 0.16.0
>
>
> I ran a job with 0 reducer on a cluster with 390 nodes.
> The mappers ran very fast.
> The jobtracker lacks behind on committing completed mapper tasks.
> The number of running mappers displayed on web UI getting bigger and bigger.
> The jos tracker eventually stopped responding to web UI.
> No progress is reported afterwards.
> Job tracker is running on a separate node.
> The job tracker process consumed 100% cpu, with vm size 1.01g (reach the heap space limit).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message