hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5487) In task processes, JobConf is unnecessarily loaded again in Limits
Date Tue, 17 Sep 2013 19:54:53 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13769854#comment-13769854

Alejandro Abdelnur commented on MAPREDUCE-5487:

On my first comment, my bad, mistakenly thought JOB_CONF_FILE was mapred-site.xml, it is job.xml,
the localized job. It is fine then.

> In task processes, JobConf is unnecessarily loaded again in Limits
> ------------------------------------------------------------------
>                 Key: MAPREDUCE-5487
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5487
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: performance, task
>    Affects Versions: 2.1.0-beta
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
>         Attachments: MAPREDUCE-5487-1.patch, MAPREDUCE-5487.patch
> Limits statically loads a JobConf, which incurs costs of reading files from disk and
parsing XML.  The contents of this JobConf are identical to the one loaded by YarnChild (before
adding job.xml as a resource).  Allowing Limits to initialize with the JobConf loaded in YarnChild
would reduce task startup time.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message