incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Jungblut (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-431) MapReduce NG integration
Date Tue, 11 Oct 2011 09:59:12 GMT

    [ https://issues.apache.org/jira/browse/HAMA-431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13124894#comment-13124894
] 

Thomas Jungblut commented on HAMA-431:
--------------------------------------

I have a problem launching the APP master.

I'm putting the classpath together via suggested code:

{noformat}
    // Assuming our classes or jars are available as local resources in the
    // working directory from which the command will be run, we need to append
    // "." to the path.
    // By default, all the hadoop specific classpaths will already be available
    // in $CLASSPATH, so we should be careful not to overwrite it.
    String classPathEnv = "$CLASSPATH:./*:";
    env.put("CLASSPATH", classPathEnv);
    amContainer.setEnvironment(env);
{noformat}

The app master gets started with following start command:

{noformat}
Start command: ${JAVA_HOME}/bin/java -cp $CLASSPATH:./*: org.apache.hama.bsp.BSPApplicationMaster
file:/home/thomasjungblut/Desktop/application_1318323647317_0012/job.xml 1><LOG_DIR>/stdout
2><LOG_DIR>/stderr
{noformat}

Which is correct, assuming that $CLASSPATH has been set properly by YARN.

But I think that it is not:

{noformat}
11/10/11 11:37:03 INFO ipc.YarnRPC: Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/avro/ipc/Server
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:169)
        at org.apache.hadoop.yarn.ipc.YarnRPC.create(YarnRPC.java:53)
        at org.apache.hama.bsp.BSPApplicationMaster.<init>(BSPApplicationMaster.java:104)
        at org.apache.hama.bsp.BSPApplicationMaster.main(BSPApplicationMaster.java:233)
Caused by: java.lang.ClassNotFoundException: org.apache.avro.ipc.Server
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
        ... 5 more
{noformat}

Is there a suggestion on how to solve this?

FYI, I'm submitting the dummy job via "yarn/bin/yarn jar xyz.jar". The generated start command
contains all the jars that should be needed. So classpath should be set properly.
                
> MapReduce NG integration
> ------------------------
>
>                 Key: HAMA-431
>                 URL: https://issues.apache.org/jira/browse/HAMA-431
>             Project: Hama
>          Issue Type: New Feature
>            Reporter: Thomas Jungblut
>            Assignee: Thomas Jungblut
>         Attachments: job_state.dot, task_phase.dot, task_state.dot
>
>
> We should take a look at how to integrate Hama's BSP Engine to Hadoop's nextGen application
platform.
> Can be currently found in the 0.23 branch.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message