giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maja Kabiljo" <majakabi...@fb.com>
Subject Review Request: GIRAPH-615: Add support for multithreaded output
Date Wed, 10 Apr 2013 00:47:46 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10389/
-----------------------------------------------------------

Review request for giraph.


Description
-------

Some output formats, like giraph-hive stuff, can write output in parallel. We should provide
an option to choose number of threads to use for storing output.

While there, I cleaned up some stuff around ExecutorService calls.


This addresses bug GIRAPH-615.
    https://issues.apache.org/jira/browse/GIRAPH-615


Diffs
-----

  giraph-core/src/main/java/org/apache/giraph/conf/GiraphConfiguration.java 90b05e3 
  giraph-core/src/main/java/org/apache/giraph/conf/GiraphConstants.java 95c9862 
  giraph-core/src/main/java/org/apache/giraph/edge/EdgeStore.java 01a67dd 
  giraph-core/src/main/java/org/apache/giraph/graph/ComputeCallable.java 51ed4f6 
  giraph-core/src/main/java/org/apache/giraph/graph/GraphTaskManager.java abca4c4 
  giraph-core/src/main/java/org/apache/giraph/utils/CallableFactory.java PRE-CREATION 
  giraph-core/src/main/java/org/apache/giraph/utils/ProgressableUtils.java 77eb49a 
  giraph-core/src/main/java/org/apache/giraph/worker/BspServiceWorker.java c20d06e 
  giraph-core/src/main/java/org/apache/giraph/worker/EdgeInputSplitsCallableFactory.java 9297ac1

  giraph-core/src/main/java/org/apache/giraph/worker/VertexInputSplitsCallableFactory.java
aebca81 

Diff: https://reviews.apache.org/r/10389/diff/


Testing
-------

mvn clean install
Tested on real application with all options (set number of output threads with an without
thread safe output), using giraph-hive.


Thanks,

Maja Kabiljo


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message