hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-848) Refactor YARN module for hadoop 2.x stable version
Date Tue, 24 Feb 2015 06:26:12 GMT

    [ https://issues.apache.org/jira/browse/HAMA-848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14334483#comment-14334483
] 

Edward J. Yoon commented on HAMA-848:
-------------------------------------

Congratz!

One minor NOTE: Resource „Ö£ocalization seems recommended - http://hortonworks.com/blog/resource-localization-in-yarn-deep-dive/
- instead of always accessing HDFS remotely.

{code}
   /**
    * Reads the configuration from the given path.
    */
-  private static Configuration getSubmitConfiguration(String path) {
+  private static Configuration getSubmitConfiguration(String path)
+      throws IOException {
     Path jobSubmitPath = new Path(path);
     Configuration jobConf = new HamaConfiguration();
-    jobConf.addResource(jobSubmitPath);
+
+    FileSystem fs = FileSystem.get(URI.create(path), jobConf);
+
+    InputStream in =fs.open(jobSubmitPath);
+    jobConf.addResource(in);
+
     return jobConf;
   }
{code}

> Refactor YARN module for hadoop 2.x stable version
> --------------------------------------------------
>
>                 Key: HAMA-848
>                 URL: https://issues.apache.org/jira/browse/HAMA-848
>             Project: Hama
>          Issue Type: Improvement
>    Affects Versions: 0.7.0
>            Reporter: Edward J. Yoon
>            Assignee: Minho Kim
>              Labels: patch
>             Fix For: 0.7.0
>
>         Attachments: HAMA-HDFS2XBuild.Patch, HAMA-YARNDraft.patch, HAMA_YARN02.patch,
HAMA_YARN03.patch, HAMA_YARN04.patch, HAMA_YARN05.patch, HAMA_YARN06.patch, HAMA_YARN07.patch,
edward.patch, x.patch
>
>
> Current yarn-module was implemented based on Hadoop 0.23.1 version. You'll need to use
the Hama RPC instead of Hadoop RPC.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message