hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: how to save memory
Date Fri, 05 Jul 2013 11:09:40 GMT

Google's Pregel is a vertex-centric programming model, and it allows
users to define communication between vertices. But, system creates
one processor per input split (partition), not vertex. Each split
contains multiple vertices.... Please don't ask "why?" :-)

Consequently, each processor need to group whole messages by vertex ID
after barrier synchronization. This message grouping requires huge
memory space.

By sorting the messages by vertex ID, we processes vertices
sequentially and avoids memory consumption.

On Fri, Jul 5, 2013 at 7:02 PM, qswm611 <qswm611@163.com> wrote:
> I  want to ask a question that why sorting message can save memory during Hama running?

Best Regards, Edward J. Yoon

View raw message