giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GIRAPH-1034) Allow IPs for Worker2Worker communication
Date Tue, 20 Oct 2015 00:44:28 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-1034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14964323#comment-14964323
] 

Hudson commented on GIRAPH-1034:
--------------------------------

FAILURE: Integrated in Giraph-trunk-Commit #1571 (See [https://builds.apache.org/job/Giraph-trunk-Commit/1571/])
GIRAPH-1034 Allow IPs for Worker2Worker communication (edunov: [http://git-wip-us.apache.org/repos/asf?p=giraph.git&a=commit&h=b735f02bde1ed684eb0d0db2ec125d26d103aced])
* giraph-core/src/main/java/org/apache/giraph/comm/netty/NettyClient.java
* giraph-core/src/main/java/org/apache/giraph/comm/MasterServer.java
* giraph-core/src/test/java/org/apache/giraph/comm/SaslConnectionTest.java
* giraph-core/src/main/java/org/apache/giraph/comm/netty/NettyWorkerServer.java
* giraph-core/src/main/java/org/apache/giraph/graph/TaskInfo.java
* giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java
* giraph-core/src/main/java/org/apache/giraph/conf/GiraphConfiguration.java
* giraph-core/src/test/java/org/apache/giraph/comm/ConnectionTest.java
* giraph-core/src/main/java/org/apache/giraph/comm/netty/NettyServer.java
* giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java
* giraph-core/src/main/java/org/apache/giraph/comm/WorkerServer.java
* giraph-core/src/test/java/org/apache/giraph/comm/RequestTest.java
* giraph-core/src/main/java/org/apache/giraph/comm/netty/NettyMasterServer.java
* giraph-core/src/test/java/org/apache/giraph/partition/SimpleRangePartitionFactoryTest.java
* giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java
* giraph-core/src/test/java/org/apache/giraph/comm/RequestFailureTest.java


> Allow IPs for Worker2Worker communication
> -----------------------------------------
>
>                 Key: GIRAPH-1034
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-1034
>             Project: Giraph
>          Issue Type: Improvement
>            Reporter: Sergey Edunov
>            Assignee: Sergey Edunov
>
> For worker2worker communication, we use O(n^2) DNS lookups, where n is the number of
workers. If network infrastructure is less reliable, combined with a large number of workers,
these DNS lookups occasionally fail. 
> Fortunately, we don't always need DNS lookups, as in many clusters hosts can communicate
with each other directly using IP addresses. Let's add a configuration option to use IP address
instead of the host name. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message