giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hassan Eslami (JIRA)" <>
Subject [jira] [Created] (GIRAPH-1150) Fixing MemoryEstimatorOracle
Date Fri, 16 Jun 2017 00:44:02 GMT
Hassan Eslami created GIRAPH-1150:

             Summary: Fixing MemoryEstimatorOracle
                 Key: GIRAPH-1150
             Project: Giraph
          Issue Type: Bug
            Reporter: Hassan Eslami
            Assignee: Hassan Eslami

* Once the OOC is at OFFLAODING state, it may offload data to disk indefinitely. We should
call GC manually and get OOC out of OFFLOADING if anything has changed in computation/communication
* Fixes a "if" statement which entirely disabled regression for memory estimation.
* Interleaving of in resetting and calculating memory estimation potentially can cause data
* The superstep count in memory estimator was not coherent throughout the calculation.
* Sometime the memory estimator's accuracy is not good. We should fall back to a threshold-based
scheme relying on a more pessimistic memory usage report (such as the one given by JVM --
memory usage includes garbage data too, but it can be used as a pessimistic estimate, as it
is currently used in ThresholdBasedOracle).

More tuning is needed for a smooth memory estimation mechanism.

This message was sent by Atlassian JIRA

View raw message