hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Minho Kim (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HAMA-848) Refactor YARN module for hadoop 2.x stable version
Date Fri, 13 Feb 2015 06:19:11 GMT

     [ https://issues.apache.org/jira/browse/HAMA-848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Minho Kim updated HAMA-848:
---------------------------
    Attachment: HAMA_YARN06.patch

Hi,
I have some questions.

I don't know why configuration xml file is changed after call reWriteSubmiConfigurations()
in BSPApplicationMaster.java.

{code}
private static void rewriteSubmitConfiguration(String path, Configuration conf)
      throws IOException {
    Path jobSubmitPath = new Path(path);
    FileSystem fs = FileSystem.get(conf);
    FSDataOutputStream out = fs.create(jobSubmitPath);
    conf.writeXml(out);
    out.close();
    fs.copyToLocalFile(false, jobSubmitPath, new Path("/root/job1.xml"));

    LOG.info("Written new configuration back to " + path);
  }
{code}

Another question is that BSPJob.writeXML() is supposed to save job configuration such as user-defined
value, user-modified value, but it was not saved.

{code}
// In BSPJobClient.java
// Write job file to BSPMaster's fs
    FSDataOutputStream out = FileSystem.create(fs, submitJobFile,
        new FsPermission(JOB_FILE_PERMISSION));

    try {
      job.writeXml(out);
    } finally {
      out.close();
    }
{code}

> Refactor YARN module for hadoop 2.x stable version
> --------------------------------------------------
>
>                 Key: HAMA-848
>                 URL: https://issues.apache.org/jira/browse/HAMA-848
>             Project: Hama
>          Issue Type: Improvement
>    Affects Versions: 0.7.0
>            Reporter: Edward J. Yoon
>            Assignee: Minho Kim
>              Labels: patch
>             Fix For: 0.7.0
>
>         Attachments: HAMA-HDFS2XBuild.Patch, HAMA-YARNDraft.patch, HAMA_YARN02.patch,
HAMA_YARN03.patch, HAMA_YARN04.patch, HAMA_YARN05.patch, HAMA_YARN06.patch, edward.patch,
x.patch
>
>
> Current yarn-module was implemented based on Hadoop 0.23.1 version. You'll need to use
the Hama RPC instead of Hadoop RPC.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message