giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Craig Muchinsky" <cmuch...@us.ibm.com>
Subject Re: Review Request 19401: GIRAPH-873 - Specialized edge stores
Date Fri, 21 Mar 2014 16:14:02 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19401/
-----------------------------------------------------------

(Updated March 21, 2014, 4:14 p.m.)


Review request for giraph.


Changes
-------

Updated with concurrency fixes


Repository: giraph-git


Description
-------

This patch converts the EdgeStore class to an interface and adds 3 impls:

SimpleEdgeStore: same behavior as original EdgeStore class
IntEdgeStore: edge store optimized for integer ids, uses fastutil primitive collections and
allows reuse of vertexId objects (which is the big performance win)
LongEdgeStore: edge store optimized for long ids, uses fastutil primitive collections and
allows reuse of vertexId objects (which is the big performance win)

The impl to use is chosen programmatically based on interrogation of the id class type. In
the future perhaps the EdgeStore impl could be specified via configuration, to allow extensibility.


Diffs (updated)
-----

  giraph-core/src/main/java/org/apache/giraph/comm/ServerData.java 39bf504 
  giraph-core/src/main/java/org/apache/giraph/edge/EdgeStore.java dd8f2a3 
  giraph-core/src/main/java/org/apache/giraph/edge/SimpleEdgeStore.java PRE-CREATION 
  giraph-core/src/main/java/org/apache/giraph/edge/primitives/IntEdgeStore.java PRE-CREATION

  giraph-core/src/main/java/org/apache/giraph/edge/primitives/LongEdgeStore.java PRE-CREATION

  giraph-core/src/main/java/org/apache/giraph/edge/primitives/package-info.java PRE-CREATION


Diff: https://reviews.apache.org/r/19401/diff/


Testing
-------

Successful "mvn clean verify" with hadoop_2 profile, and 4B vertex 5B edge graph tested on
18 node 432 core cluster.


Thanks,

Craig Muchinsky


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message