Return-Path: X-Original-To: apmail-incubator-giraph-user-archive@minotaur.apache.org Delivered-To: apmail-incubator-giraph-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3E54F98CC for ; Tue, 3 Apr 2012 21:01:45 +0000 (UTC) Received: (qmail 22320 invoked by uid 500); 3 Apr 2012 21:01:45 -0000 Delivered-To: apmail-incubator-giraph-user-archive@incubator.apache.org Received: (qmail 22291 invoked by uid 500); 3 Apr 2012 21:01:45 -0000 Mailing-List: contact giraph-user-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: giraph-user@incubator.apache.org Delivered-To: mailing list giraph-user@incubator.apache.org Received: (qmail 22283 invoked by uid 99); 3 Apr 2012 21:01:45 -0000 Received: from minotaur.apache.org (HELO minotaur.apache.org) (140.211.11.9) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Apr 2012 21:01:45 +0000 Received: from localhost (HELO carynmba.thefacebook.com) (127.0.0.1) (smtp-auth username aching, mechanism plain) by minotaur.apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Apr 2012 21:01:44 +0000 Message-ID: <4F7B6539.1030209@apache.org> Date: Tue, 03 Apr 2012 14:01:45 -0700 From: Avery Ching User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:11.0) Gecko/20120313 Thunderbird/11.0 MIME-Version: 1.0 To: giraph-user@incubator.apache.org Subject: Re: Exceptions when establishing RPC References: In-Reply-To: Content-Type: multipart/alternative; boundary="------------080306010604010209030303" This is a multi-part message in MIME format. --------------080306010604010209030303 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit If you're using one master and one slave, you need to do -w 1. Did you see any error about the RPC server starting up? Avery On 4/3/12 1:37 PM, Robert Davis wrote: > Hello, > > I was trying to run Giraph on two machines (one master and one slave) > but kept getting exceptions when establishing RPC to the slave > machine. Does anybody has any ideas what's going wrong here? I am > running the test with following parameters. > > hadoop jar target/giraph-0.2-SNAPSHOT-jar-with-dependencies.jar > org.apache.giraph.benchmark.PageRankBenchmark -e 10 -s 2 -v -V 2000 -w 2 > > Thanks, > Robert > > 12/04/03 01:35:01 DEBUG comm.BasicRPCCommunications: > startPeerConnectionThread: hostname > ec2-107-20-19-131.compute-1.amazonaws.com > , port 30001 > 12/04/03 01:35:01 DEBUG comm.BasicRPCCommunications: > startPeerConnectionThread: Connecting to > Worker(hostname=ec2-107-20-19-131.compute-1.amazonaws.com > , MRpartition=1, > port=30001), addr = ec2-107-20-19-131.compute-1.amazonaws.com:30001 > if outMsgMap > (null) == null > 12/04/03 01:35:11 WARN comm.BasicRPCCommunications: > connectAllRPCProxys: Failed on attempt 1 of 5 to connect to > (id=0,cur=Worker(hostname=ec2-107-20-19-131.compute-1.amazonaws.com > , MRpartition=1, > port=30001),prev=null,ckpt_file=null) > java.net.ConnectException: Call to > ec2-107-20-19-131.compute-1.amazonaws.com:30001 > failed on > connection exception: java.net.ConnectException: Connection refused > at org.apache.hadoop.ipc.Client.wrapException(Client.java:1095) > at org.apache.hadoop.ipc.Client.call(Client.java:1071) > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225) > at $Proxy3.getProtocolVersion(Unknown Source) > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396) > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:370) > at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:420) > at > org.apache.giraph.comm.RPCCommunications$1.run(RPCCommunications.java:194) > at > org.apache.giraph.comm.RPCCommunications$1.run(RPCCommunications.java:190) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083) > at > org.apache.giraph.comm.RPCCommunications.getRPCProxy(RPCCommunications.java:188) > at > org.apache.giraph.comm.RPCCommunications.getRPCProxy(RPCCommunications.java:58) > at > org.apache.giraph.comm.BasicRPCCommunications.startPeerConnectionThread(BasicRPCCommunications.java:678) > at > org.apache.giraph.comm.BasicRPCCommunications.connectAllRPCProxys(BasicRPCCommunications.java:622) > at > org.apache.giraph.comm.BasicRPCCommunications.setup(BasicRPCCommunications.java:583) > at > org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:555) > at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:474) > at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:646) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083) > at org.apache.hadoop.mapred.Child.main(Child.java:249) > Caused by: java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567) > at > org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) > at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:656) > at > org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:434) > at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:560) > at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:184) > at org.apache.hadoop.ipc.Client.getConnection(Client.java:1202) > at org.apache.hadoop.ipc.Client.call(Client.java:1046) > ... 25 more > --------------080306010604010209030303 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit If you're using one master and one slave, you need to do -w 1.  Did you see any error about the RPC server starting up?

Avery

On 4/3/12 1:37 PM, Robert Davis wrote:
Hello,

I was trying to run Giraph on two machines (one master and one slave) but kept getting exceptions when establishing RPC to the slave machine. Does anybody has any ideas what's going wrong here? I am running the test with following parameters.

hadoop jar target/giraph-0.2-SNAPSHOT-jar-with-dependencies.jar org.apache.giraph.benchmark.PageRankBenchmark -e 10 -s 2 -v -V 2000 -w 2

Thanks,
Robert

12/04/03 01:35:01 DEBUG comm.BasicRPCCommunications: startPeerConnectionThread: hostname ec2-107-20-19-131.compute-1.amazonaws.com, port 30001
12/04/03 01:35:01 DEBUG comm.BasicRPCCommunications: startPeerConnectionThread: Connecting to Worker(hostname=ec2-107-20-19-131.compute-1.amazonaws.com, MRpartition=1, port=30001), addr = ec2-107-20-19-131.compute-1.amazonaws.com:30001 if outMsgMap (null) == null 
12/04/03 01:35:11 WARN comm.BasicRPCCommunications: connectAllRPCProxys: Failed on attempt 1 of 5 to connect to (id=0,cur=Worker(hostname=ec2-107-20-19-131.compute-1.amazonaws.com, MRpartition=1, port=30001),prev=null,ckpt_file=null)
java.net.ConnectException: Call to ec2-107-20-19-131.compute-1.amazonaws.com:30001 failed on connection exception: java.net.ConnectException: Connection refused
at org.apache.hadoop.ipc.Client.wrapException(Client.java:1095)
at org.apache.hadoop.ipc.Client.call(Client.java:1071)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
at $Proxy3.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:370)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:420)
at org.apache.giraph.comm.RPCCommunications$1.run(RPCCommunications.java:194)
at org.apache.giraph.comm.RPCCommunications$1.run(RPCCommunications.java:190)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
at org.apache.giraph.comm.RPCCommunications.getRPCProxy(RPCCommunications.java:188)
at org.apache.giraph.comm.RPCCommunications.getRPCProxy(RPCCommunications.java:58)
at org.apache.giraph.comm.BasicRPCCommunications.startPeerConnectionThread(BasicRPCCommunications.java:678)
at org.apache.giraph.comm.BasicRPCCommunications.connectAllRPCProxys(BasicRPCCommunications.java:622)
at org.apache.giraph.comm.BasicRPCCommunications.setup(BasicRPCCommunications.java:583)
at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:555)
at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:474)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:646)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:656)
at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:434)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:560)
at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:184)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1202)
at org.apache.hadoop.ipc.Client.call(Client.java:1046)
... 25 more


--------------080306010604010209030303--