hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1345) JobTracker is slowed down because it forks subprocesses to do a df command
Date Wed, 30 Dec 2009 01:32:29 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12795216#action_12795216
] 

Todd Lipcon commented on MAPREDUCE-1345:
----------------------------------------

Is this covered by HADOOP-5958?

> JobTracker is slowed down because it forks subprocesses to do a df command
> --------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1345
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1345
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: dhruba borthakur
>            Assignee: Scott Chen
>
> The JobTracker periodically does a df on the local directories. It forks a shell a shell
to run a df command. The creation of the separate process is very slow because the process
address space is copied by the OS on every subprocess creation. This becomes worse when the
JT is configured to use a large heap space. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message