hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-704) Optimization of memory usage during message processing
Date Mon, 18 Feb 2013 06:05:15 GMT

    [ https://issues.apache.org/jira/browse/HAMA-704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13580438#comment-13580438

Edward J. Yoon commented on HAMA-704:

{quote}The design should get things to work on 16GB 400 node cluster as well as a 4GB 15 node

Suraj, I don't want to consider such extreme scenarios at the moment.

Suraj: so I wanted to have a brief discussion about the whole matter 
there is no escape .. we have to think about persisting the vertices outiside main memory
4:51 PM 
Suraj: that is why Thomas was thinking of making things Disk based.

As I told you before, each Task loads 256MB vertices and uses 30GB memory. I don't know why
you think "disk-based vertices" is helpful? Memory usage is continuously increased by vertex.setValue()?

> Optimization of memory usage during message processing
> ------------------------------------------------------
>                 Key: HAMA-704
>                 URL: https://issues.apache.org/jira/browse/HAMA-704
>             Project: Hama
>          Issue Type: Improvement
>          Components: graph
>            Reporter: Edward J. Yoon
>            Assignee: Edward J. Yoon
>            Priority: Critical
>             Fix For: 0.6.1
>         Attachments: hama-704_v05.patch, localdisk.patch, mytest.patch, patch.txt, patch.txt,
> <vertex, message> map seems consume a lot of memory. We should figure out an efficient
way to reduce memory.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message