hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Jungblut <thomas.jungb...@gmail.com>
Subject Re: Messages in Memory
Date Wed, 20 Feb 2013 16:26:49 GMT
Just analysed this:
https://issues.apache.org/jira/browse/HAMA-704#comment-13582292

I'm seeing no memory leak with pagerank, this time it must be in your code
:-P

2013/2/20 Thomas Jungblut <thomas.jungblut@gmail.com>

> That looks indeed like a leak. The problem is that this could be anywhere,
> even in your user code.
> Will brew something together to reproduce this...
>
>
> 2013/2/20 Francisco Sanches <sanchesbanhos@gmail.com>
>
>> Hi Thomas,
>>
>> I observed this by measuring the amount of memory consumed in each
>> superstep. I create all the vectors of the superstep distance 0, this
>> superstep has a jump in memory usage. The Problem is that the use not
>> stabilizes, memory consumption always increases. I think it's messages,
>> because in eclipse I run on a only computer I can run a graph with 10,000
>> vertices in a cluster with 15 machines can not run a graph of 40 000
>> vertices.
>>
>>
>> 2013/2/20 Thomas Jungblut <thomas.jungblut@gmail.com>
>>
>> > >
>> > > From what I could see all the messages that are sent remain in
>> > > memory even if the superstep taking over.
>> > >
>> >
>> > How did you measure this? Can you boil it down to a structure that holds
>> > that messages? Have you profiled it?
>> >
>>
>>
>>
>> --
>> Francisco Sanches
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message