giraph-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From clau...@apache.org
Subject git commit: updated refs/heads/trunk to 5e758b8
Date Tue, 07 May 2013 23:51:28 GMT
Updated Branches:
  refs/heads/trunk 5922de147 -> 5e758b849


GIRAPH-622


Project: http://git-wip-us.apache.org/repos/asf/giraph/repo
Commit: http://git-wip-us.apache.org/repos/asf/giraph/commit/5e758b84
Tree: http://git-wip-us.apache.org/repos/asf/giraph/tree/5e758b84
Diff: http://git-wip-us.apache.org/repos/asf/giraph/diff/5e758b84

Branch: refs/heads/trunk
Commit: 5e758b849e817d846c5a39b57bb07b72eaf40f7d
Parents: 5922de1
Author: Claudio Martella <claudio@apache.org>
Authored: Wed May 8 01:51:13 2013 +0200
Committer: Claudio Martella <claudio@apache.org>
Committed: Wed May 8 01:51:13 2013 +0200

----------------------------------------------------------------------
 src/site/xdoc/ooc.xml |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/giraph/blob/5e758b84/src/site/xdoc/ooc.xml
----------------------------------------------------------------------
diff --git a/src/site/xdoc/ooc.xml b/src/site/xdoc/ooc.xml
index 2887a0d..a113d3b 100644
--- a/src/site/xdoc/ooc.xml
+++ b/src/site/xdoc/ooc.xml
@@ -42,7 +42,7 @@ under the License.
     <section name="Out-of-core Messages">
       <p>When running out-of-core messages, Giraph will keep only a limited number
of messages in memory, while the others will be stored to local disk(s). This feature can
be enabled with parameter "giraph.useOutOfCoreMessages=true" (disabled by default), while
the number of messages is controlled by parameter "giraph.maxMessagesInMemory=N" (with default
value 1000000). With this feature, Giraph will keep in memory the incoming messages into an
in-memory store. When the store exceeds the chosen number of messages, the content of the
store will be spilled to disk, and a new empty in-memory store will be instantiated. This
process produces a number of files on disk, depending on the number of messages produced during
a superstep. During the vertex computation the files will be read sequentially, and the messages
for each vertex will be concatenated and fed to the vertex. Both for reading and writing,
files are accessed sequentially.
       </p>
-        <p>Also out-of-core messages can take advantage of multiple disks, as parameter
"giraph.messagesDirectory" (with default "_bsp/_messages/") can accept a comma-separated list
of paths. It is possible to control the buffers used for i/o with parameter "giraph.messagesBufferSize=#Bytes"
(with default value 8192).
+      <p>Also out-of-core messages can take advantage of multiple disks, as parameter
"giraph.messagesDirectory" (with default "_bsp/_messages/") can accept a comma-separated list
of paths. It is possible to control the buffers used for i/o with parameter "giraph.messagesBufferSize=#Bytes"
(with default value 8192).
       </p>
     </section>
       <p>It is difficult to decide a general policy to use out-of-core capabilities,
as it depends on the behavior of the algorithm and the input graph. The exact number of partitions
and messages to keep in memory depends on the cluster capabilities, the number of messages
produced per superstep, and number of active vertices per superstep. Moreover, it depends
on the type and size of vertex values and messages. For example, algorithms such as Belief
Propagation tend to keep large vertex values, while algorithms such as clique computations
tend to send large messages along. Hence, it depends on your algorithm what feature to rely
on more. 


Mime
View raw message