giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suijian Zhou <suijian.z...@gmail.com>
Subject Re: How to set more zooKeeper nodes in giraph.
Date Tue, 08 Apr 2014 21:52:50 GMT
Hi, Claudio,
  Thanks a lot, I'm using the zookeeper coming from giraph, I did not
install zookeeper seperately. By the way, how to set the runtime parameter
of giraph.zkServerCount? If possible, I'd like to modify the "negotiated
timeout = 600000" too, but how? Thanks.

  Best Regards,
  Suijian



2014-04-08 16:24 GMT-05:00 Claudio Martella <claudio.martella@gmail.com>:

> you don't need to recompile it, you can set it runtime by setting giraph.zkServerCount
> accordingly.
> but are you trying to get giraph to start multiple instances of zookeeper
> in multiple nodes, or do you want giraph to use multiple of your existing
> zookeepers?
>
>
> On Tue, Apr 8, 2014 at 11:17 PM, Suijian Zhou <suijian.zhou@gmail.com>wrote:
>
>> Hi,
>>   Does anybody know how to set more zooKeeper nodes in giraph? I tried to
>> modify
>> ZOOKEEPER_SERVER_COUNT in file:
>> giraph-core/target/munged/main/org/apache/giraph/conf/GiraphConstants.java
>>
>> but recompilation of giraph shows no effect at all( giraph seems always
>> use 1 zookeeper node?) and when it failed(e.g, due to timeout), the client
>> could not connect and finally the giraph job failed too. It's also strange
>> that although I see "negotiated timeout = 600000" which means the session
>> is supposed to be running for 10 minutes , but why the job failed to
>> connect to it only after ~1 minutes?
>>
>> 14/04/08 15:58:18 INFO zookeeper.ZooKeeper: Initiating client connection,
>> connectString=compute-0-23.local:22181 sessionTimeout=60000
>> watcher=org.apache.giraph.job.JobProgressTracker@69b28a51
>> 14/04/08 15:58:18 INFO mapred.JobClient: Running job:
>> job_201404081444_0011
>> 14/04/08 15:58:18 INFO zookeeper.ClientCnxn: Opening socket connection to
>> server compute-0-23.local/10.1.255.231:22181. Will not attempt to
>> authenticate using SASL (unknown error)
>> 14/04/08 15:58:18 INFO zookeeper.ClientCnxn: Socket connection
>> established to compute-0-23.local/10.1.255.231:22181, initiating session
>> 14/04/08 15:58:18 INFO zookeeper.ClientCnxn: Session establishment
>> complete on server compute-0-23.local/10.1.255.231:22181, sessionid =
>> 0x14543222b640009, negotiated timeout = 600000
>> ........
>> ........
>> 14/04/08 15:59:48 INFO job.JobProgressTracker: Data from 8 workers -
>> Compute superstep 2: 0 out of 4847571 vertices computed; 0 out of 64
>> partitions computed; min free memory on worker 8 - 152.48MB, average
>> 217.58MB
>> 14/04/08 15:59:51 INFO zookeeper.ClientCnxn: Unable to read additional
>> data from server sessionid 0x14543222b640009, likely server has closed
>> socket, closing socket connection and attempting reconnect
>> 14/04/08 15:59:52 INFO zookeeper.ClientCnxn: Opening socket connection to
>> server compute-0-23.local/10.1.255.231:22181. Will not attempt to
>> authenticate using SASL (unknown error)
>> 14/04/08 15:59:52 WARN zookeeper.ClientCnxn: Session 0x14543222b640009
>> for server null, unexpected error, closing socket connection and attempting
>> reconnect
>> java.net.ConnectException: Connection refused
>>
>>   Best Regards,
>>   Suijian
>>
>>
>>
>>
>>
>
>
> --
>    Claudio Martella
>
>

Mime
View raw message