hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: Load Graph into memory ?
Date Wed, 10 Jul 2013 12:19:30 GMT
Please try to use DiskVerticesInfo and DiskQueue. It is in my TODO
list but have no time for checking them.

And, if you have an interested in solving problems and contributing
your codes to Apache Hama, Please feel free to submit your patches.
:-)

On Wed, Jul 10, 2013 at 9:04 PM, Kostas Xirog <k0stasx1rog@gmail.com> wrote:
> Thanks for the quick reply,I understand.
> As far as the memory problem goes, is there any way for me to fix it? I'm
> running the program on an 8 core machine  with 8 gb of RAM ...
>
> Thanks in advance,
> Kostas X.
>
>
> On Wed, Jul 10, 2013 at 2:54 PM, Edward J. Yoon <edwardyoon@apache.org>wrote:
>
>> >> hama to create the graph for this data. I want to load 100 million
>> records
>> >> into the system. Is there a way I can do that only once and then run
>>
>> P.S., you will face the memory problem if you don't have large cluster.
>>
>> On Wed, Jul 10, 2013 at 8:33 PM, Edward J. Yoon <edwardyoon@apache.org>
>> wrote:
>> > If you mean the reuse of already partitioned data, yes. But not support
>> yet.
>> >
>> > If you mean the reuse of vertices in memory (skip the loading phase),
>> > no. Since the system terminates the program automatically when no more
>> > updates occur, you cannot initialize the vertex values and cannot run
>> > same algorithm again. However, I think we can consider new API for
>> > initializing and reusing vertices.
>> >
>> > On Wed, Jul 10, 2013 at 4:31 PM, Kostas Xirog <k0stasx1rog@gmail.com>
>> wrote:
>> >> Hello,
>> >>
>> >>   I've written this application that creates a graph and runs a query on
>> >> this graph.I want to test this application with very large amounts of
>> data.
>> >> The graph is created using data that's structured into records.
>> >> When I have 5 million records for instance, it takes about 5 minutes for
>> >> hama to create the graph for this data. I want to load 100 million
>> records
>> >> into the system. Is there a way I can do that only once and then run
>> >> programs on the existing graph?
>> >>
>> >> Thank you in advance,
>> >> Kostas X.
>> >
>> >
>> >
>> > --
>> > Best Regards, Edward J. Yoon
>> > @eddieyoon
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>



-- 
Best Regards, Edward J. Yoon
@eddieyoon

Mime
View raw message