incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Jungblut (Issue Comment Edited) (JIRA)" <j...@apache.org>
Subject [jira] [Issue Comment Edited] (HAMA-454) Add Zookeeper as synchronization service
Date Fri, 21 Oct 2011 16:34:32 GMT

    [ https://issues.apache.org/jira/browse/HAMA-454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13132803#comment-13132803
] 

Thomas Jungblut edited comment on HAMA-454 at 10/21/11 4:34 PM:
----------------------------------------------------------------

Zookeeper now works (theoretically).

Sometimes I still have shutdown issues in the  ApplicationMaster:

{noformat}

11/10/21 18:30:57 INFO server.NIOServerCnxn: NIOServerCnxn factory exited run method
11/10/21 18:30:57 INFO server.PrepRequestProcessor: PrepRequestProcessor exited loop!
11/10/21 18:30:57 INFO server.SyncRequestProcessor: SyncRequestProcessor exited!
11/10/21 18:30:57 INFO server.FinalRequestProcessor: shutdown of request processor complete
11/10/21 18:30:57 INFO server.FinalRequestProcessor: shutdown of request processor complete
11/10/21 18:30:57 WARN jmx.MBeanRegistry: Failed to unregister MBean InMemoryDataTree
11/10/21 18:30:57 WARN jmx.MBeanRegistry: Error during unregister
javax.management.InstanceNotFoundException: org.apache.ZooKeeperService:name0=StandaloneServer_port-1,name1=InMemoryDataTree
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1094)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:415)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:403)
        at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:506)
        at org.apache.zookeeper.jmx.MBeanRegistry.unregister(MBeanRegistry.java:94)
        at org.apache.zookeeper.jmx.MBeanRegistry.unregister(MBeanRegistry.java:111)
        at org.apache.zookeeper.server.ZooKeeperServer.unregisterJMX(ZooKeeperServer.java:421)
        at org.apache.zookeeper.server.ZooKeeperServer.shutdown(ZooKeeperServer.java:414)
        at org.apache.zookeeper.server.NIOServerCnxn$Factory.shutdown(NIOServerCnxn.java:323)
        at org.apache.zookeeper.server.ZooKeeperServerMain.shutdown(ZooKeeperServerMain.java:125)
        at org.apache.hama.zookeeper.QuorumPeer$ShutdownableZooKeeperServerMain.shutdown(QuorumPeer.java:407)
        at org.apache.hama.zookeeper.QuorumPeer$ShutdownableZooKeeperServerMain.shutdownZookeeperMain(QuorumPeer.java:402)
        at org.apache.hama.bsp.sync.zookeeper.ZooKeeperSyncServerImpl.stopServer(ZooKeeperSyncServerImpl.java:82)
        at org.apache.hama.bsp.sync.SyncServerRunner.stop(SyncServerRunner.java:47)
        at org.apache.hama.bsp.BSPApplicationMaster.cleanup(BSPApplicationMaster.java:243)
        at org.apache.hama.bsp.BSPApplicationMaster.main(BSPApplicationMaster.java:279)
{noformat}

Besides this everything works.
Even the RPC Synchronization works, but we have to manually build core module with Hadoop
23.0.

Unit tests complete as well:

{noformat}
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Apache Hama parent POM ............................ SUCCESS [3.121s]
[INFO] Apache Hama Core .................................. SUCCESS [11.107s]
[INFO] Apache Hama Graph Package ......................... SUCCESS [2.572s]
[INFO] Apache Hama Examples .............................. SUCCESS [25.469s]
[INFO] Apache Hama YARN .................................. SUCCESS [46.075s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:30.192s
[INFO] Finished at: Fri Oct 21 18:29:10 CEST 2011
[INFO] Final Memory: 35M/353M
[INFO] ------------------------------------------------------------------------

{noformat}
                
      was (Author: thomas.jungblut):
    Zookeeper now works (theoretically).

Sometimes I still have shutdown issues in the  ApplicationMaster:

{noformat}

11/10/21 18:30:57 INFO server.NIOServerCnxn: NIOServerCnxn factory exited run method
11/10/21 18:30:57 INFO server.PrepRequestProcessor: PrepRequestProcessor exited loop!
11/10/21 18:30:57 INFO server.SyncRequestProcessor: SyncRequestProcessor exited!
11/10/21 18:30:57 INFO server.FinalRequestProcessor: shutdown of request processor complete
11/10/21 18:30:57 INFO server.FinalRequestProcessor: shutdown of request processor complete
11/10/21 18:30:57 WARN jmx.MBeanRegistry: Failed to unregister MBean InMemoryDataTree
11/10/21 18:30:57 WARN jmx.MBeanRegistry: Error during unregister
javax.management.InstanceNotFoundException: org.apache.ZooKeeperService:name0=StandaloneServer_port-1,name1=InMemoryDataTree
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1094)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:415)
        at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:403)
        at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:506)
        at org.apache.zookeeper.jmx.MBeanRegistry.unregister(MBeanRegistry.java:94)
        at org.apache.zookeeper.jmx.MBeanRegistry.unregister(MBeanRegistry.java:111)
        at org.apache.zookeeper.server.ZooKeeperServer.unregisterJMX(ZooKeeperServer.java:421)
        at org.apache.zookeeper.server.ZooKeeperServer.shutdown(ZooKeeperServer.java:414)
        at org.apache.zookeeper.server.NIOServerCnxn$Factory.shutdown(NIOServerCnxn.java:323)
        at org.apache.zookeeper.server.ZooKeeperServerMain.shutdown(ZooKeeperServerMain.java:125)
        at org.apache.hama.zookeeper.QuorumPeer$ShutdownableZooKeeperServerMain.shutdown(QuorumPeer.java:407)
        at org.apache.hama.zookeeper.QuorumPeer$ShutdownableZooKeeperServerMain.shutdownZookeeperMain(QuorumPeer.java:402)
        at org.apache.hama.bsp.sync.zookeeper.ZooKeeperSyncServerImpl.stopServer(ZooKeeperSyncServerImpl.java:82)
        at org.apache.hama.bsp.sync.SyncServerRunner.stop(SyncServerRunner.java:47)
        at org.apache.hama.bsp.BSPApplicationMaster.cleanup(BSPApplicationMaster.java:243)
        at org.apache.hama.bsp.BSPApplicationMaster.main(BSPApplicationMaster.java:279)
{noformat}

Besides this everything works.
Even the RPC Synchronization works, but we have to manually build core module with Hadoop
23.0.
                  
> Add Zookeeper as synchronization service
> ----------------------------------------
>
>                 Key: HAMA-454
>                 URL: https://issues.apache.org/jira/browse/HAMA-454
>             Project: Hama
>          Issue Type: Sub-task
>            Reporter: Thomas Jungblut
>            Assignee: Thomas Jungblut
>         Attachments: HAMA-454.patch, HAMA-454.patch, HAMA-454_v2.patch
>
>
> We should use Zookeeper instead of our own implementation.
> Additionally we should use the plain BSPPeerImpl in YARN to reduce duplicate code.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message