giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alessandro Presta" <alessan...@fb.com>
Subject Re: Review Request: GIRAPH-200: remove hadoop RPC and keep just netty
Date Thu, 11 Oct 2012 01:33:33 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7521/
-----------------------------------------------------------

(Updated Oct. 11, 2012, 1:33 a.m.)


Review request for giraph.


Changes
-------

Fixed giraph-formats-contrib.


Description
-------

- Removed Hadoop RPC.
- Simplified Vertex API: since messages are not stored in Vertex anymore, now the standard
way to define your own edge data structure is by implementing setEdges()/setNeighbors(). It's
still possible to override initialize() if more customization is needed.


This addresses bug GIRAPH-200.
    https://issues.apache.org/jira/browse/GIRAPH-200


Diffs (updated)
-----

  /trunk/giraph-formats-contrib/src/main/java/org/apache/giraph/io/hcatalog/HCatalogVertexInputFormat.java
1396858 
  /trunk/giraph-formats-contrib/src/test/java/org/apache/giraph/io/accumulo/edgemarker/AccumuloEdgeInputFormat.java
1396858 
  /trunk/giraph-formats-contrib/src/test/java/org/apache/giraph/io/hbase/edgemarker/TableEdgeInputFormat.java
1396858 
  /trunk/pom.xml 1396858 
  /trunk/src/main/java/org/apache/giraph/GiraphConfiguration.java 1396858 
  /trunk/src/main/java/org/apache/giraph/bsp/CentralizedServiceWorker.java 1396858 
  /trunk/src/main/java/org/apache/giraph/comm/BasicRPCCommunications.java 1396858 
  /trunk/src/main/java/org/apache/giraph/comm/CommunicationsInterface.java 1396858 
  /trunk/src/main/java/org/apache/giraph/comm/RPCCommunications.java 1396858 
  /trunk/src/main/java/org/apache/giraph/comm/SecureRPCCommunications.java 1396858 
  /trunk/src/main/java/org/apache/giraph/comm/VertexIdMessages.java 1396858 
  /trunk/src/main/java/org/apache/giraph/comm/VertexIdMessagesList.java 1396858 
  /trunk/src/main/java/org/apache/giraph/comm/VertexList.java 1396858 
  /trunk/src/main/java/org/apache/giraph/comm/WorkerServer.java 1396858 
  /trunk/src/main/java/org/apache/giraph/comm/messages/SimpleMessageStore.java 1396858 
  /trunk/src/main/java/org/apache/giraph/comm/messages/package-info.java 1396858 
  /trunk/src/main/java/org/apache/giraph/comm/netty/NettyServer.java 1396858 
  /trunk/src/main/java/org/apache/giraph/comm/netty/NettyWorkerClient.java 1396858 
  /trunk/src/main/java/org/apache/giraph/comm/package-info.java 1396858 
  /trunk/src/main/java/org/apache/giraph/examples/LongDoubleFloatDoubleTextInputFormat.java
1396858 
  /trunk/src/main/java/org/apache/giraph/examples/NormalizingLongDoubleFloatDoubleTextInputFormat.java
1396858 
  /trunk/src/main/java/org/apache/giraph/examples/SimpleMutateGraphVertex.java 1396858 
  /trunk/src/main/java/org/apache/giraph/examples/SimplePageRankVertex.java 1396858 
  /trunk/src/main/java/org/apache/giraph/examples/SimpleSuperstepVertex.java 1396858 
  /trunk/src/main/java/org/apache/giraph/graph/BspServiceMaster.java 1396858 
  /trunk/src/main/java/org/apache/giraph/graph/BspServiceWorker.java 1396858 
  /trunk/src/main/java/org/apache/giraph/graph/EdgeListVertex.java 1396858 
  /trunk/src/main/java/org/apache/giraph/graph/GraphMapper.java 1396858 
  /trunk/src/main/java/org/apache/giraph/graph/HashMapVertex.java 1396858 
  /trunk/src/main/java/org/apache/giraph/graph/IntIntNullIntVertex.java 1396858 
  /trunk/src/main/java/org/apache/giraph/graph/LongDoubleFloatDoubleEdgeListVertex.java 1396858

  /trunk/src/main/java/org/apache/giraph/graph/LongDoubleFloatDoubleVertex.java 1396858 
  /trunk/src/main/java/org/apache/giraph/graph/LongDoubleNullDoubleVertex.java 1396858 
  /trunk/src/main/java/org/apache/giraph/graph/MutableVertex.java 1396858 
  /trunk/src/main/java/org/apache/giraph/graph/SimpleMutableVertex.java 1396858 
  /trunk/src/main/java/org/apache/giraph/graph/SimpleVertex.java 1396858 
  /trunk/src/main/java/org/apache/giraph/graph/Vertex.java 1396858 
  /trunk/src/main/java/org/apache/giraph/graph/VertexResolver.java 1396858 
  /trunk/src/main/java/org/apache/giraph/graph/WorkerInfo.java 1396858 
  /trunk/src/main/java/org/apache/giraph/hadoop/BspPolicyProvider.java 1396858 
  /trunk/src/main/java/org/apache/giraph/hadoop/BspTokenSelector.java 1396858 
  /trunk/src/main/java/org/apache/giraph/hadoop/package-info.java 1396858 
  /trunk/src/main/java/org/apache/giraph/io/PseudoRandomVertexInputFormat.java 1396858 
  /trunk/src/main/java/org/apache/giraph/io/TextVertexInputFormat.java 1396858 
  /trunk/src/test/java/org/apache/giraph/comm/RPCCommunicationsTest.java 1396858 
  /trunk/src/test/java/org/apache/giraph/comm/RequestTest.java 1396858 
  /trunk/src/test/java/org/apache/giraph/examples/SimpleShortestPathsVertexTest.java 1396858

  /trunk/src/test/java/org/apache/giraph/examples/SimpleTriangleClosingVertexTest.java 1396858

  /trunk/src/test/java/org/apache/giraph/examples/TryMultiIpcBindingPortsTest.java PRE-CREATION

  /trunk/src/test/java/org/apache/giraph/examples/TryMultiRpcBindingPortsTest.java 1396858

  /trunk/src/test/java/org/apache/giraph/graph/TestEdgeListVertex.java 1396858 
  /trunk/src/test/java/org/apache/giraph/graph/TestIntIntNullIntVertex.java 1396858 
  /trunk/src/test/java/org/apache/giraph/graph/partition/TestPartitionStores.java 1396858

  /trunk/src/test/java/org/apache/giraph/io/TestTextDoubleDoubleAdjacencyListVertexInputFormat.java
1396858 

Diff: https://reviews.apache.org/r/7521/diff/


Testing
-------

- mvn verify (with various profiles)
- pseudo-distributed mode tests


Thanks,

Alessandro Presta


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message