giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alessandro Presta (JIRA)" <j...@apache.org>
Subject [jira] [Created] (GIRAPH-249) Move part of the graph out-of-core when memory is low
Date Thu, 12 Jul 2012 13:26:34 GMT
Alessandro Presta created GIRAPH-249:
----------------------------------------

             Summary: Move part of the graph out-of-core when memory is low
                 Key: GIRAPH-249
                 URL: https://issues.apache.org/jira/browse/GIRAPH-249
             Project: Giraph
          Issue Type: Improvement
            Reporter: Alessandro Presta
            Assignee: Alessandro Presta


There has been some talk about Giraph's scaling limitations due to keeping the whole graph
and messages in RAM.
We need to investigate methods to fall back to disk when running out of memory, while gracefully
degrading performance.

This issue is for graph storage. Messages should probably be a separate issue, although the
interplay between the two is crucial.

We should also discuss what are our primary goals here: completing a job (albeit slowly) instead
of failing when the graph is too big, while still encouraging memory optimizations and high-memory
clusters; or restructuring Giraph to be as efficient as possible in disk mode, making it almost
a standard way of operating.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message