giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Edunov (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (GIRAPH-882) List of zookeeper connection strings is trimmed by Hadoop counters.
Date Wed, 29 Jun 2016 17:43:17 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15355531#comment-15355531
] 

Sergey Edunov edited comment on GIRAPH-882 at 6/29/16 5:42 PM:
---------------------------------------------------------------

Hi Dongjin, 

Are you available for chat sometime this week? I'm trying to figure out how this diff relates
to GIRAPH-1068. There we removed an option to start multiple zookeepers from the giraph itself.
It is still possible to connect to external zookeeper quorum though. 
To understand what needs to be done here, can you explain your use case? If I understand correctly,
here you create 5 zookeeper instances from the Giraph job. What is the reason, and why don't
you use single zookeper?  


was (Author: edunov):
Hi Donngjin, 

Are you available for chat sometime this week? I'm trying to figure out how this diff relates
to GIRAPH-1068. There we removed an option to start multiple zookeepers from the giraph itself.
It is still possible to connect to external zookeeper quorum though. 
To understand what needs to be done here, can you explain your use case? If I understand correctly,
here you create 5 zookeeper instances from the Giraph job. What is the reason, and why don't
you use single zookeper?  

> List of zookeeper connection strings is trimmed by Hadoop counters.
> -------------------------------------------------------------------
>
>                 Key: GIRAPH-882
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-882
>             Project: Giraph
>          Issue Type: Bug
>          Components: zookeeper
>    Affects Versions: 1.1.0, 1.2.0
>            Reporter: Lukas Nalezenec
>         Attachments: GIRAPH-882-rev2.patch, GIRAPH-882-rev3.patch, GIRAPH-882.patch,
testrun.log
>
>
> We are running job with quorum of 3 zookeepers. Each serves has got long name (turing452.fi.callan.de:22181).
Connection strings are stored to Hadoop Counters (for example: turing452.fi.callan.de:22181,turing124.fi.callan.de:22181,turing488.fi.callan.de:22181)
but since name of counter is limited to ~63 character the connection string is trimmed (turing452.fi.callan.de:22181,turing124.fi.callan.de:22181,turin).
> 14/03/18 23:44:41 INFO zookeeper.ZooKeeper: Client environment:user.name=hadoop
> 14/03/18 23:44:41 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=turing452.fi.callan.de:22181,turing124.fi.callan.de:22181,turin
sessionTimeout=60000 
> Exception in thread "main" java.net.UnknownHostException: turin
> 	at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
> 	at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901)
> 	at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293)
> 	at java.net.InetAddress.getAllByName0(InetAddress.java:1246)
> 	at java.net.InetAddress.getAllByName(InetAddress.java:1162)
> 	at java.net.InetAddress.getAllByName(InetAddress.java:1098)
> 	at org.apache.zookeeper.client.StaticHostProvider.<init>(StaticHostProvider.java:60)
> 	at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:445)
> 	at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:380)
> 	at org.apache.giraph.zk.ZooKeeperExt.<init>(ZooKeeperExt.java:114)
> 	at org.apache.giraph.job.JobProgressTracker.<init>(JobProgressTracker.java:69)
> 	at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:255)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message