Return-Path: X-Original-To: apmail-incubator-hama-user-archive@minotaur.apache.org Delivered-To: apmail-incubator-hama-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CAA4D4CC0 for ; Tue, 5 Jul 2011 16:22:15 +0000 (UTC) Received: (qmail 55785 invoked by uid 500); 5 Jul 2011 16:22:15 -0000 Delivered-To: apmail-incubator-hama-user-archive@incubator.apache.org Received: (qmail 55727 invoked by uid 500); 5 Jul 2011 16:22:15 -0000 Mailing-List: contact hama-user-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hama-user@incubator.apache.org Delivered-To: mailing list hama-user@incubator.apache.org Received: (qmail 55717 invoked by uid 99); 5 Jul 2011 16:22:14 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 05 Jul 2011 16:22:14 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [80.190.178.166] (HELO mail.digital.tis.bz.it) (80.190.178.166) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 05 Jul 2011 16:22:07 +0000 Received: from tyler.unibz.it (unknown [46.18.27.4]) by mail.digital.tis.bz.it (Mailserver) with ESMTPSA id 5D934123A001 for ; Tue, 5 Jul 2011 18:21:45 +0200 (CEST) Message-ID: <4E133A1A.1020502@tis.bz.it> Date: Tue, 05 Jul 2011 18:21:46 +0200 From: Claudio Martella User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.18) Gecko/20110616 Lightning/1.0b2 Thunderbird/3.1.11 MIME-Version: 1.0 To: hama-user@incubator.apache.org Subject: Re: on the compatibility of hama References: <4E132A61.9090507@tis.bz.it> In-Reply-To: <4E132A61.9090507@tis.bz.it> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org Ok, apparently there are two options to set the ZK client port, i set it correctly through zookeeper.client.port. Now I'm getting this error at groom start: localhost: Exception in thread "regionservernull" java.lang.NoSuchMethodError: org.apache.hadoop.filecache.DistributedCache.purgeCache(Lorg/apache/hadoo= p/conf/Configuration;)V localhost: at org.apache.hama.bsp.GroomServer.initialize(GroomServer.java:202) localhost: at org.apache.hama.bsp.GroomServer.run(GroomServer.java:54= 1) localhost: at java.lang.Thread.run(Thread.java:662) I'm afraid there are mismatches between hadoop 0.20.2 as hama is built on (and shipped with) and the hadoop from CDH I'm using right now. The method has been probably removed. Any suggestion? I can easily comment the call out, apparently it's bad practice for the user to call the purgeCache directly. On 7/5/11 5:14 PM, Claudio Martella wrote: > Hello, > > I'm trying to install hama-0.2.0 in pseudo-distributed mode. As I have > already HBase, which I'm going to read from, installed on my system, I'= m > basically trying to get hama to work with CDH3. So basically: > > - hbase 0.90.1-cdh3u0 > - hadoop 0.20.2-cdh3u0 > - zookeeper 3.3.3-cdh3u0 > > > I configured hadoop in pseudo-distributed mode and launched the namenod= e > and the datanode. I've configured zookpeeper with default configs. I'm > also running hbase and from the client shell i can connect to ZK. > > This is hbase's shell zkdump: > > base(main):004:0> zk_dump > HBase is rooted at /hbase > Master address: localhost:60000 > Region server holding ROOT: localhost:60020 > Region servers: > localhost:60020 > Quorum Server Statistics: > localhost:2181 > Zookeeper version: 3.3.3-cdh3u0--1, built on 03/25/2011 23:05 GMT > Clients: > /127.0.0.1:35176[1](queued=3D0,recved=3D190,sent=3D199) > /0:0:0:0:0:0:0:1:57645[1](queued=3D0,recved=3D163,sent=3D164) > /127.0.0.1:35163[1](queued=3D0,recved=3D201,sent=3D213) > /127.0.0.1:53414[0](queued=3D0,recved=3D1,sent=3D0) > /0:0:0:0:0:0:0:1:38233[1](queued=3D0,recved=3D14,sent=3D14) > /0:0:0:0:0:0:0:1:57647[1](queued=3D0,recved=3D161,sent=3D162) > =20 > Latency min/avg/max: 0/0/47 > Received: 731 > Sent: 753 > Outstanding: 0 > Zxid: 0x28 > Mode: standalone > Node count: 12 > > > So apparently ZK is up and running. I've started hama with these > hama-site.xml: > > > > hama.zookeeper.property.clientPort > 2181 > > > fs.default.name > hdfs://localhost:8020 > > > > But i get these errors at startup of groom: > > STARTUP_MSG: build =3D -r 81256ad0f2e4ab2bd34b04f53d25a6c23686dd14; > compiled by 'hudson' on Fri Mar 25 19:56:23 PDT 2011 > ************************************************************/ > 2011-07-05 17:09:49,566 INFO org.apache.hama.bsp.GroomServer: groom sta= rt > 2011-07-05 17:09:49,575 INFO org.apache.hama.bsp.GroomServer: > [Ljava.lang.String;@20b9b538 > 2011-07-05 17:09:49,575 INFO org.apache.hama.bsp.GroomServer: > /tmp/hadoop-cmartella/bsp/local > 2011-07-05 17:09:49,742 INFO org.apache.hadoop.ipc.Server: Starting > Socket Reader #1 for port 61000 > 2011-07-05 17:09:49,744 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: > Initializing RPC Metrics with hostName=3DBSPPeer, port=3D61000 > 2011-07-05 17:09:49,775 INFO > org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC > Metrics with hostName=3DBSPPeer, port=3D61000 > 2011-07-05 17:09:49,777 INFO org.apache.hadoop.ipc.Server: IPC Server > Responder: starting > 2011-07-05 17:09:49,778 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 0 on 61000: starting > 2011-07-05 17:09:49,778 INFO org.apache.hama.bsp.BSPPeer: BSPPeer > address:rose.inf.unibz.it port:61000 > 2011-07-05 17:09:49,777 INFO org.apache.hadoop.ipc.Server: IPC Server > listener on 61000: starting > 2011-07-05 17:09:49,807 INFO org.apache.zookeeper.ZooKeeper: Client > environment:zookeeper.version=3D3.3.3-cdh3u0--1, built on 03/25/2011 23= :05 GMT > 2011-07-05 17:09:49,807 INFO org.apache.zookeeper.ZooKeeper: Client > environment:host.name=3Drose.inf.unibz.it > 2011-07-05 17:09:49,807 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.version=3D1.6.0_22 > 2011-07-05 17:09:49,807 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.vendor=3DSun Microsystems Inc. > 2011-07-05 17:09:49,807 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.home=3D/usr/lib/jvm/java-6-sun-1.6.0.22/jre > 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.class.path=3D/data/cmartella/graffiti/hama-0.2.0/bin/.= =2E/conf:/usr/lib/jvm/java-6-sun/lib/tools.jar:/data/cmartella/graffiti/h= ama-0.2.0/bin/../hama-0.2.0-examples.jar:/data/cmartella/graffiti/hama-0.= 2.0/bin/../hama-0.2.0-test.jar:/data/cmartella/graffiti/hama-0.2.0/bin/..= /hama-0.2.0.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/GroboTesti= ngJUnit-1.2.1-core.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/com= mons-cli-1.2.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/commons-l= ogging-1.0.4.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/commons-l= ogging-api-1.0.4.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/hadoo= p-core-0.20.2-cdh3u0.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/j= unit-3.8.1.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/log4j-1.2.1= 5.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/zookeeper-3.3.3-cdh3= u0.jar:/data/cmartella/graffiti/hama-0.2.0/bin/../lib/jsp-2.1/*.jar > 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.library.path=3D/usr/lib/jvm/java-6-sun-1.6.0.22/jre/li= b/amd64/server:/usr/lib/jvm/java-6-sun-1.6.0.22/jre/lib/amd64:/usr/lib/jv= m/java-6-sun-1.6.0.22/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/= lib64:/lib64:/lib:/usr/lib > 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.io.tmpdir=3D/tmp > 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:java.compiler=3D > 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:os.name=3DLinux > 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:os.arch=3Damd64 > 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:os.version=3D2.6.32-27-server > 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:user.name=3Dcmartella > 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:user.home=3D/home/students/cmartella > 2011-07-05 17:09:49,808 INFO org.apache.zookeeper.ZooKeeper: Client > environment:user.dir=3D/data/cmartella/graffiti/hama-0.2.0 > 2011-07-05 17:09:49,809 INFO org.apache.zookeeper.ZooKeeper: Initiating= > client connection, connectString=3Dlocalhost:21810 sessionTimeout=3D300= 0 > watcher=3Dorg.apache.hama.bsp.BSPPeer@217f242c > 2011-07-05 17:09:49,820 INFO org.apache.zookeeper.ClientCnxn: Opening > socket connection to server localhost/127.0.0.1:21810 > 2011-07-05 17:09:49,824 WARN org.apache.zookeeper.ClientCnxn: Session > 0x0 for server null, unexpected error, closing socket connection and > attempting reconnect > java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 119) > 2011-07-05 17:09:49,938 ERROR org.apache.hama.bsp.BSPPeer: > 2011-07-05 17:09:50,237 INFO org.apache.zookeeper.ClientCnxn: Opening > socket connection to server localhost/0:0:0:0:0:0:0:1:21810 > 2011-07-05 17:09:50,238 WARN org.apache.zookeeper.ClientCnxn: Session > 0x0 for server null, unexpected error, closing socket connection and > attempting reconnect > java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 119) > 2011-07-05 17:09:49,938 ERROR org.apache.hama.bsp.BSPPeer: > 2011-07-05 17:09:50,237 INFO org.apache.zookeeper.ClientCnxn: Opening > socket connection to server localhost/0:0:0:0:0:0:0:1:21810 > 2011-07-05 17:09:50,238 WARN org.apache.zookeeper.ClientCnxn: Session > 0x0 for server null, unexpected error, closing socket connection and > attempting reconnect > java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at > sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574) > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1= 119) > 2011-07-05 17:09:50,338 ERROR org.apache.hama.bsp.BSPPeer: > org.apache.zookeeper.KeeperException$ConnectionLossException: > KeeperErrorCode =3D ConnectionLoss for /bsp > 2011-07-05 17:09:50,341 INFO org.apache.hadoop.ipc.Server: Starting > Socket Reader #1 for port 50000 > 2011-07-05 17:09:50,341 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: > Initializing RPC Metrics with hostName=3DGroomServer, port=3D50000 > 2011-07-05 17:09:50,341 INFO > org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC > Metrics with hostName=3DGroomServer, port=3D50000 > 2011-07-05 17:09:50,342 INFO org.apache.hadoop.ipc.Server: IPC Server > Responder: starting > 2011-07-05 17:09:50,343 INFO org.apache.hadoop.ipc.Server: IPC Server > listener on 50000: starting > 2011-07-05 17:09:50,343 INFO org.apache.hama.bsp.GroomServer: Worker rp= c > server --> rose.inf.unibz.it:50000 > 2011-07-05 17:09:50,343 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 0 on 50000: starting > 2011-07-05 17:09:50,345 INFO org.apache.hadoop.ipc.Server: Starting > Socket Reader #1 for port 47963 > 2011-07-05 17:09:50,345 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: > Initializing RPC Metrics with hostName=3DGroomServer, port=3D47963 > 2011-07-05 17:09:50,345 INFO > org.apache.hadoop.ipc.metrics.RpcDetailedMetrics: Initializing RPC > Metrics with hostName=3DGroomServer, port=3D47963 > 2011-07-05 17:09:50,347 INFO org.apache.hadoop.ipc.Server: IPC Server > Responder: starting > 2011-07-05 17:09:50,347 INFO org.apache.hadoop.ipc.Server: IPC Server > listener on 47963: starting > 2011-07-05 17:09:50,347 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 0 on 47963: starting > 2011-07-05 17:09:50,347 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 1 on 47963: starting > 2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 2 on 47963: starting > 2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 3 on 47963: starting > 2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 4 on 47963: starting > 2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 5 on 47963: starting > 2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 6 on 47963: starting > 2011-07-05 17:09:50,348 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 7 on 47963: starting > 2011-07-05 17:09:50,349 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 8 on 47963: starting > 2011-07-05 17:09:50,349 INFO org.apache.hadoop.ipc.Server: IPC Server > handler 9 on 47963: starting > 2011-07-05 17:09:50,349 INFO org.apache.hama.bsp.GroomServer: > GroomServer up at: localhost/127.0.0.1:47963 > 2011-07-05 17:09:50,349 INFO org.apache.hama.bsp.GroomServer: Starting > groom: groomd_rose.inf.unibz.it_61000 > 2011-07-05 17:09:50,351 INFO org.apache.hama.GroomServerRunner: > SHUTDOWN_MSG: > > As there were compatibility problems, I substituted the hadoop-core and= > zookeper jars in lib/ with those shipped by CDH. > Apparenty groom is trying to connect to ZK on (hama's default) 21810 > although I've set the port to ZK's default 2181. > > Any idea why this is happening? > > Best, > Claudio > --=20 Claudio Martella Free Software & Open Technologies Analyst TIS innovation park Via Siemens 19 | Siemensstr. 19 39100 Bolzano | 39100 Bozen Tel. +39 0471 068 123 Fax +39 0471 068 129 claudio.martella@tis.bz.it http://www.tis.bz.it Short information regarding use of personal data. According to Section 13= of Italian Legislative Decree no. 196 of 30 June 2003, we inform you tha= t we process your personal data in order to fulfil contractual and fiscal= obligations and also to send you information regarding our services and = events. Your personal data are processed with and without electronic mean= s and by respecting data subjects' rights, fundamental freedoms and digni= ty, particularly with regard to confidentiality, personal identity and th= e right to personal data protection. At any time and without formalities = you can write an e-mail to privacy@tis.bz.it in order to object the proce= ssing of your personal data for the purpose of sending advertising materi= als and also to exercise the right to access personal data and other righ= ts referred to in Section 7 of Decree 196/2003. The data controller is TI= S Techno Innovation Alto Adige, Siemens Street n. 19, Bolzano. You can fi= nd the complete information on the web site www.tis.bz.it.