giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hassan Eslami (JIRA)" <>
Subject [jira] [Resolved] (GIRAPH-1025) Flow-control mechanism improving out-of-core feature
Date Tue, 26 Apr 2016 18:06:13 GMT


Hassan Eslami resolved GIRAPH-1025.
    Resolution: Fixed

Addressed in GIRAPH-1027

> Flow-control mechanism improving out-of-core feature
> ----------------------------------------------------
>                 Key: GIRAPH-1025
>                 URL:
>             Project: Giraph
>          Issue Type: Improvement
>          Components: graph
>            Reporter: Hassan Eslami
>            Assignee: Hassan Eslami
> Current implementation of out-of-core mechanism may fail due to message flooding in applications
without message combiner. In such applications the rate at which messages arrive may be much
more than the rate of read/write to disk in OOC mechanism. In this scenario, the OOC mechanism
will get OOM as it cannot keep up with storing incoming messages in-memory.
> One might say this problem may also appear in input-superstep as we have input vertices
and edges going around as messages, but note that in input superstep the rate at which a worker
receive messages (vertices and edges) are almost the same as the rate of reading from disk,
and that is almost the same as the rate at which offloading data to disk happens. However,
this flow-control mechanism can be generalized for input superstep too. At this point, this
generalization does not seem necessary though.

This message was sent by Atlassian JIRA

View raw message