hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Jungblut (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-704) Optimization of memory usage during message processing
Date Fri, 22 Feb 2013 12:30:12 GMT

    [ https://issues.apache.org/jira/browse/HAMA-704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584224#comment-13584224
] 

Thomas Jungblut commented on HAMA-704:
--------------------------------------

*First profiling*

> First Superstep after messages are completely send and in their outgoing queue.

Strong reachable Objects consume 637 mb. (strong refers that they can't be collected because
they are reachable)
Messages consume the most of the memory, if we subtract them out of the memory we have:
24mb for the DiskVerticesInfo, most of it consumed by the long[] that stores the offset. 

Messages consume a bit over 600mb, the rest of it is data structure overhead.

Great stuff. Now we have to get the messaging going ;-)
                
> Optimization of memory usage during message processing
> ------------------------------------------------------
>
>                 Key: HAMA-704
>                 URL: https://issues.apache.org/jira/browse/HAMA-704
>             Project: Hama
>          Issue Type: Improvement
>          Components: graph
>            Reporter: Edward J. Yoon
>            Assignee: Edward J. Yoon
>            Priority: Critical
>             Fix For: 0.6.1
>
>         Attachments: HAMA-704_1337.patch, HAMA-704.patch-v1, hama-704_v05.patch, HAMA-704-v2.patch,
localdisk.patch, mytest.patch, patch.txt, patch.txt, removeMsgMap.patch
>
>
> <vertex, message> map seems consume a lot of memory. We should figure out an efficient
way to reduce memory.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message