giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maja Kabiljo" <majakabi...@fb.com>
Subject Re: Review Request: GIRAPH-372: Write worker addresses to Zookeeper; move addresses and resolution to NettyClient
Date Wed, 17 Oct 2012 22:01:29 GMT


> On Oct. 17, 2012, 9:55 p.m., Avery Ching wrote:
> > +1!  Thanks for all the changes Maja!

Thanks for the review, Avery!

Forgot to hit publish along the way... Eli about your comments:
- I wanted to change WorkerInfo name, but it appears in too many places so I thought it's
better to do it in a separate Jira.
- Addresses are now written together with partitions owners to the same znode.
- I agree there is more cleanup to be done. Writing WorkerInfos from PartitionOwners is redundant.
But again, we can open another issue for that and discuss additional improvements there.


- Maja


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7610/#review12540
-----------------------------------------------------------


On Oct. 17, 2012, 9:52 p.m., Maja Kabiljo wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7610/
> -----------------------------------------------------------
> 
> (Updated Oct. 17, 2012, 9:52 p.m.)
> 
> 
> Review request for giraph.
> 
> 
> Description
> -------
> 
> In preparation for GIRAPH-273, we need to have addresses of all workers available, so
I write them to Zookeeper along with the master address.
> 
> Since address resoultion is needed in both NettyWorkerClient and NettyMasterClient, I
moved that to NettyClient, and also added a map taskId->address in there. Now NettyMasterClient
and NettyWorkerClient don't need to take care of InetSocketAddresses, just task ids which
they want to send messages to.
> 
> 
> This addresses bug GIRAPH-372.
>     https://issues.apache.org/jira/browse/GIRAPH-372
> 
> 
> Diffs
> -----
> 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/bsp/CentralizedService.java
1399356 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java
1399356 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/MasterClient.java
1399356 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/WorkerClient.java
1399356 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/netty/NettyClient.java
1399356 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/netty/NettyMasterClient.java
1399356 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/netty/NettyMasterClientServer.java
1399356 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClient.java
1399356 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClientRequestProcessor.java
1399356 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/graph/AddressesAndPartitionsWritable.java
PRE-CREATION 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/graph/BspService.java
1399356 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/graph/BspServiceMaster.java
1399356 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/graph/BspServiceWorker.java
1399356 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/graph/InputSplitsCallable.java
1399356 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/graph/WorkerInfo.java
1399356 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/main/java/org/apache/giraph/utils/WritableUtils.java
1399356 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/test/java/org/apache/giraph/comm/ConnectionTest.java
1399356 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/test/java/org/apache/giraph/comm/RequestFailureTest.java
1399356 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/test/java/org/apache/giraph/comm/RequestTest.java
1399356 
>   http://svn.apache.org/repos/asf/giraph/trunk/giraph/src/test/java/org/apache/giraph/comm/SaslConnectionTest.java
1399356 
> 
> Diff: https://reviews.apache.org/r/7610/diff/
> 
> 
> Testing
> -------
> 
> Passes mvn verify
> 
> 
> Thanks,
> 
> Maja Kabiljo
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message