giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Edunov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GIRAPH-882) List of zookeeper connection strings is trimmed by Hadoop counters.
Date Fri, 01 Jul 2016 16:38:11 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359227#comment-15359227
] 

Sergey Edunov commented on GIRAPH-882:
--------------------------------------

I believe that if you set up external zookeeper quorum this code isn't even gets called. In
this case, all access to the zookeeper goes through ZooKeeperExt. Check for example GraphTaskManager.java,
these lines:

    String serverPortList = conf.getZookeeperList();
    if (serverPortList.isEmpty()) {
      if (startZooKeeperManager()) {
        return; // ZK connect/startup failed
      }
    } else {
      createZooKeeperCounter(serverPortList);
    }

The first "if" is the only entry point into the ZooKeeperManager as far as I can see. And
we don't go there if external zookeeper quorum is set. 

> List of zookeeper connection strings is trimmed by Hadoop counters.
> -------------------------------------------------------------------
>
>                 Key: GIRAPH-882
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-882
>             Project: Giraph
>          Issue Type: Bug
>          Components: zookeeper
>    Affects Versions: 1.1.0, 1.2.0
>            Reporter: Lukas Nalezenec
>         Attachments: GIRAPH-882-rev2.patch, GIRAPH-882-rev3.patch, GIRAPH-882.patch,
testrun.log
>
>
> We are running job with quorum of 3 zookeepers. Each serves has got long name (turing452.fi.callan.de:22181).
Connection strings are stored to Hadoop Counters (for example: turing452.fi.callan.de:22181,turing124.fi.callan.de:22181,turing488.fi.callan.de:22181)
but since name of counter is limited to ~63 character the connection string is trimmed (turing452.fi.callan.de:22181,turing124.fi.callan.de:22181,turin).
> 14/03/18 23:44:41 INFO zookeeper.ZooKeeper: Client environment:user.name=hadoop
> 14/03/18 23:44:41 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=turing452.fi.callan.de:22181,turing124.fi.callan.de:22181,turin
sessionTimeout=60000 
> Exception in thread "main" java.net.UnknownHostException: turin
> 	at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
> 	at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901)
> 	at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293)
> 	at java.net.InetAddress.getAllByName0(InetAddress.java:1246)
> 	at java.net.InetAddress.getAllByName(InetAddress.java:1162)
> 	at java.net.InetAddress.getAllByName(InetAddress.java:1098)
> 	at org.apache.zookeeper.client.StaticHostProvider.<init>(StaticHostProvider.java:60)
> 	at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:445)
> 	at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:380)
> 	at org.apache.giraph.zk.ZooKeeperExt.<init>(ZooKeeperExt.java:114)
> 	at org.apache.giraph.job.JobProgressTracker.<init>(JobProgressTracker.java:69)
> 	at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:255)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message