hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-848) Refactor YARN module for hadoop 2.x stable version
Date Sat, 28 Feb 2015 01:07:06 GMT

    [ https://issues.apache.org/jira/browse/HAMA-848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14341135#comment-14341135
] 

Edward J. Yoon commented on HAMA-848:
-------------------------------------

Here's few comments:

1. I can't build TRUNK w/ this patch for Hadoop 1.0. The reason is that YarnConfiguration
class does not exist in Hadoop 1.0 version. My suggestion is that you override submitJobInternal()
method in yarn bsp client class.

{code}
% mvn clean install -Phadoop1 -Dhadoop.version=1.x.x
{code}

2. AFAIK, new instance of subclass also can be created by reflection utils. Instead of separating
NullInputFormat into multi classes, let's try to figure out what's the problem one more time.
And, below change is meaningless I think.

{code}
Index: core/src/main/java/org/apache/hama/bsp/BSPPeerImpl.java
===================================================================
--- core/src/main/java/org/apache/hama/bsp/BSPPeerImpl.java	(revision 1659147)
+++ core/src/main/java/org/apache/hama/bsp/BSPPeerImpl.java	(working copy)
@@ -249,6 +249,7 @@
     try {
       if (splitClass != null) {
         inputSplit = (InputSplit) ReflectionUtils.newInstance(
+            //Class.forName(splitClass), getConfiguration());
             getConfiguration().getClassByName(splitClass), getConfiguration());
       }
     } catch (ClassNotFoundException exp) {
{code}

3. If possible, let's unblock startSyncServer() method.

{code}
     // start our synchronization service
-    startSyncServer();
+    //startSyncServer();
{code}

> Refactor YARN module for hadoop 2.x stable version
> --------------------------------------------------
>
>                 Key: HAMA-848
>                 URL: https://issues.apache.org/jira/browse/HAMA-848
>             Project: Hama
>          Issue Type: Improvement
>    Affects Versions: 0.7.0
>            Reporter: Edward J. Yoon
>            Assignee: Minho Kim
>              Labels: patch
>             Fix For: 0.7.0
>
>         Attachments: HAMA-848.patch, HAMA-HDFS2XBuild.Patch, HAMA-YARNDraft.patch, HAMA_YARN02.patch,
HAMA_YARN03.patch, HAMA_YARN04.patch, HAMA_YARN05.patch, HAMA_YARN06.patch, HAMA_YARN07.patch,
edward.patch, x.patch
>
>
> Current yarn-module was implemented based on Hadoop 0.23.1 version. You'll need to use
the Hama RPC instead of Hadoop RPC.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message