incubator-giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Claudio Martella (Commented) (JIRA)" <>
Subject [jira] [Commented] (GIRAPH-104) Save half of maximum memory used from messaging
Date Tue, 13 Dec 2011 22:29:30 GMT


Claudio Martella commented on GIRAPH-104:

supposing the messaging pattern doesn't change between superstep 6 and superstep 8 :)

this looks like a great improvement, great work. I went through the review, frankly quite
quickly, and it looks very good.

I'll check it out better tomorrow and will +1.
> Save half of maximum memory used from messaging
> -----------------------------------------------
>                 Key: GIRAPH-104
>                 URL:
>             Project: Giraph
>          Issue Type: Improvement
>            Reporter: Avery Ching
>            Assignee: Avery Ching
>            Priority: Critical
>         Attachments: GIRAPH-104.diff
> Currently, the amount of memory that Giraph uses for messaging is huge.  This JIRA will
reduce the messaging memory by half and provide periodic updates of memory for debugging.
 Details are below:
> Refactored RandomMessageBenchmark to an internal vertex class.  Added aggregators to
RandomMessagesBenchmark to track bytes, messages, and time for the messaging.  Adjusted the
postSuperstep() to be called after the flush() for more accurate timings.
> Added periodic minute updates for message flushing (which can take a while, especially
on the memory benchmark).  This helps to see how progress is going and gives an ETA.
> Memory optimizations include:
> - Clear the message list after computation 
> - Free vertex messages on the source as the flush is going on 
> - TreeMap -> HashMap for VertexMutations
> - Sizing the ArrayList properly in transientInMessages

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message