incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-454) Add Zookeeper as synchronization service
Date Tue, 25 Oct 2011 00:58:32 GMT

    [ https://issues.apache.org/jira/browse/HAMA-454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13134669#comment-13134669
] 

Edward J. Yoon commented on HAMA-454:
-------------------------------------

Here's my quick test results:

{code}

root@Cnode1:/usr/local/src/hama-trunk# core/bin/hama jar examples/target/hama-examples-0.4.0-incubating-SNAPSHOT.jar
pi
11/10/25 09:51:06 INFO bsp.BSPJobClient: Running job: job_201110250950_0001
11/10/25 09:51:09 INFO bsp.BSPJobClient: Current supersteps number: 0
11/10/25 09:51:15 INFO bsp.BSPJobClient: Current supersteps number: 1
11/10/25 09:51:18 INFO bsp.BSPJobClient: The total number of supersteps: 1
Estimated value of PI is 3.1417666666666673
Job Finished in 12.914 seconds
root@Cnode1:/usr/local/src/hama-trunk# core/bin/stop-bspd.sh
stopping bspmaster
hnode3: stopping groom
hnode2: stopping groom
hnode1: stopping groom
hnode4: stopping groom
hnode1: stopping zookeeper
root@Cnode1:/usr/local/src/hama-trunk# svn stat
?       HAMA-454_v3.patch
M       core/conf/hama-env.sh
M       core/conf/hama-site.xml
M       core/conf/groomservers
root@Cnode1:/usr/local/src/hama-trunk# patch -p0 < HAMA-454_v3.patch
patching file core/src/main/java/org/apache/hama/zookeeper/QuorumPeer.java
patching file core/src/main/java/org/apache/hama/bsp/BSPPeerImpl.java
patching file core/src/main/java/org/apache/hama/bsp/sync/SyncServer.java
patching file core/src/main/java/org/apache/hama/bsp/sync/zookeeper/ZooKeeperSyncClientImpl.java
patching file core/src/main/java/org/apache/hama/bsp/sync/zookeeper/ZooKeeperSyncServerImpl.java
patching file core/src/main/java/org/apache/hama/bsp/sync/rpc/RPCSyncClientImpl.java
patching file core/src/main/java/org/apache/hama/bsp/sync/rpc/RPCSyncServerImpl.java
patching file core/src/main/java/org/apache/hama/bsp/sync/SyncServerRunner.java
patching file yarn/src/main/java/org/apache/hama/bsp/BSPClient.java
patching file yarn/src/main/java/org/apache/hama/bsp/BSPApplicationMaster.java
patching file yarn/src/main/java/org/apache/hama/bsp/YARNBSPJob.java
patching file yarn/src/main/java/org/apache/hama/bsp/sync/SyncServer.java
patching file yarn/src/main/java/org/apache/hama/bsp/sync/SyncServerImpl.java
patching file yarn/src/main/java/org/apache/hama/bsp/YARNBSPPeerImpl.java
patching file yarn/src/main/java/org/apache/hama/bsp/JobImpl.java
patching file yarn/src/main/java/org/apache/hama/bsp/YarnSerializePrinting.java
patching file yarn/src/main/java/org/apache/hama/bsp/BSPRunner.java

root@Cnode1:/usr/local/src/hama-trunk# mvn clean install package -Dmaven.test.skip=true
....

root@Cnode1:/usr/local/src/hama-trunk# core/bin/start-bspd.sh
hnode1: starting zookeeper, logging to /usr/local/src/hama-trunk/core/bin/../logs/hama-root-zookeeper-Cnode1.out
starting bspmaster, logging to /usr/local/src/hama-trunk/core/bin/../logs/hama-root-bspmaster-Cnode1.out
2011-10-25 09:54:04.190::INFO:  Logging to STDERR via org.mortbay.log.StdErrLog
2011-10-25 09:54:04.232::INFO:  jetty-6.1.14
hnode3: starting groom, logging to /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-Cnode3.out
hnode1: starting groom, logging to /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-Cnode1.out
hnode4: starting groom, logging to /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-Cnode4.out
hnode2: starting groom, logging to /usr/local/src/hama-trunk/core/bin/../logs/hama-root-groom-Cnode2.out

root@Cnode1:/usr/local/src/hama-trunk# core/bin/hama jar examples/target/hama-examples-0.4.0-incubating-SNAPSHOT.jar
pi
11/10/25 09:54:29 INFO bsp.BSPJobClient: Running job: job_201110250954_0001
11/10/25 09:54:32 INFO bsp.BSPJobClient: Current supersteps number: 0
............................................................

GroomServer LOG:

2011-10-25 09:55:48,130 INFO org.apache.hama.bsp.GroomServer: Starting groom: cnode1.ucloud:50000
2011-10-25 09:56:15,283 WARN org.apache.hama.bsp.GroomServer: Error initializing attempt_201110250955_0001_000002_0:
java.lang.NullPointerException
        at org.apache.hama.bsp.GroomServer.localizeJob(GroomServer.java:548)
        at org.apache.hama.bsp.GroomServer.startNewTask(GroomServer.java:488)
        at org.apache.hama.bsp.GroomServer.access$000(GroomServer.java:76)
        at org.apache.hama.bsp.GroomServer$DispatchTasksHandler.handle(GroomServer.java:165)
        at org.apache.hama.bsp.GroomServer$Instructor.run(GroomServer.java:210)

2011-10-25 09:56:15,315 WARN org.apache.hama.bsp.GroomServer: Error initializing attempt_201110250955_0001_000008_0:
java.lang.NullPointerException
        at org.apache.hama.bsp.GroomServer.localizeJob(GroomServer.java:548)
        at org.apache.hama.bsp.GroomServer.startNewTask(GroomServer.java:488)
        at org.apache.hama.bsp.GroomServer.access$000(GroomServer.java:76)
        at org.apache.hama.bsp.GroomServer$DispatchTasksHandler.handle(GroomServer.java:165)
        at org.apache.hama.bsp.GroomServer$Instructor.run(GroomServer.java:210)

2011-10-25 09:56:18,151 INFO org.apache.hama.bsp.TaskRunner: Start building BSPPeer process.

{code}
                
> Add Zookeeper as synchronization service
> ----------------------------------------
>
>                 Key: HAMA-454
>                 URL: https://issues.apache.org/jira/browse/HAMA-454
>             Project: Hama
>          Issue Type: Sub-task
>            Reporter: Thomas Jungblut
>            Assignee: Thomas Jungblut
>         Attachments: HAMA-454.patch, HAMA-454.patch, HAMA-454_v2.patch, HAMA-454_v3.patch
>
>
> We should use Zookeeper instead of our own implementation.
> Additionally we should use the plain BSPPeerImpl in YARN to reduce duplicate code.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message