hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4066) To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value
Date Mon, 26 Mar 2012 07:18:30 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238157#comment-13238157
] 

Harsh J commented on MAPREDUCE-4066:
------------------------------------

Good catch xieguiming.

Could you post a patch for this as well?

There are other spots that may need defaults too:
{code}
./hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java:
     doneDirPrefix = conf.get(MRJobConfig.MR_AM_STAGING_DIR)
./hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/jobhistory/JobHistoryUtils.java:
     doneDirPrefix = conf.get(MRJobConfig.MR_AM_STAGING_DIR)
./hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/v2/util/MRApps.java:
       conf.get(MRJobConfig.MR_AM_STAGING_DIR) + 
{code}

In case you are willing to provide a patch, please make sure to address the following:
* Define the default property and value in mapred-default.xml
* Define a constant DEFAULT_MR_AM_STAGING_DIR in MRJobConfig, set to same value as above
* Use the above constant in the code where you'll add the defaults to those three spots above
(MRApps and JobHistoryUtils).

Thanks! :)
                
> To get "yarn.app.mapreduce.am.staging-dir" value, should set the default value
> ------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-4066
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4066
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: job submission, mrv2
>    Affects Versions: 0.23.1
>         Environment: client is windows eclipse, server is suse
>            Reporter: xieguiming
>            Priority: Minor
>
> when submit the job use the windows eclipse, and the yarn.app.mapreduce.am.staging-dir
value is null.
> {code:title=MRApps.java|borderStyle=solid}
>   public static Path getStagingAreaDir(Configuration conf, String user) {
>     return new Path(
>         conf.get(MRJobConfig.MR_AM_STAGING_DIR) + 
>         Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
>   }
> {code}
> should modify to:
> {code:title=MRApps.java|borderStyle=solid}
>   public static Path getStagingAreaDir(Configuration conf, String user) {
>     return new Path(
>         conf.get(MRJobConfig.MR_AM_STAGING_DIR,"/tmp/hadoop-yarn/staging") + 
>         Path.SEPARATOR + user + Path.SEPARATOR + STAGING_CONSTANT);
>   }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message