giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hassan Eslami <hsn.esl...@gmail.com>
Subject Re: Giraph out-of-core feature is not helping!
Date Fri, 11 Dec 2015 05:57:59 GMT
OOC features (both for graph and messages) are under active development at
the moment. The previous version (1.0.0 and 1.1.0) did have several issues
(mostly performance issues and also falling short interacting with
stop-the-world GC pauses while OOC tries to spill data to disk). A new
mechanism for OOC is already pushed into the code base. However, the new
mechanism does not work properly for OOC messages. An entirely new approach
to OOC is under development and soon will be available in the code base.

Best,
Hassan

On Thu, Dec 10, 2015 at 8:55 PM, Khaled Ammar <khaled.ammar@gmail.com>
wrote:

> Thank you Roman,
>
> This is interesting. I think I used OOC on the older Giraph version 1.0.0.
> Does the new version introduce new bugs?
>
> I checked the archive, but could not see relevant information. Does any
> one have experience with OOC on 1.1.0 or 1.2 ?!!
>
> Thanks,
> -Khaled
>
>
> On Fri, Dec 4, 2015 at 8:56 PM, Roman Shaposhnik <roman@shaposhnik.org>
> wrote:
>
>> I remember seeing a discussion that discouraged users from
>> utilizing OOC. I am vague on the details but you can try
>> searching the archives.
>>
>> Thanks,
>> Roman.
>>
>> On Fri, Dec 4, 2015 at 4:34 PM, Khaled Ammar <khaled.ammar@gmail.com>
>> wrote:
>> > Hi,
>> >
>> > I am using Giraph-1.1.0 to do large graph processing. I was trying to
>> do a
>> > hashMin (WCC) algorithm on a large graph but it failed with out of
>> memory
>> > error. I thought the out-of-core option may help, but it did not.
>> >
>> > Is there any advice about how to enable out-of-core processing?
>> >
>> > I followed this URL: http://giraph.apache.org/ooc.html
>> >
>> > --
>> > Thanks,
>> > -Khaled
>>
>
>
>
> --
> Thanks,
> -Khaled
>

Mime
View raw message