Mailing-List: contact dev-help@giraph.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@giraph.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CACEJUc7Q9_1VZeQYpkCUWRMFJO5AA_piDFxUHXRVEk62VRmw3Q@mail.gmail.com>
References: 
 <CACEJUc7Q9_1VZeQYpkCUWRMFJO5AA_piDFxUHXRVEk62VRmw3Q@mail.gmail.com>
From: Hassan Eslami <hsn.eslami@gmail.com>
Date: Fri, 11 Dec 2015 00:18:47 -0600
Message-ID: 
 <CAH1LQfeRA6sPQ_o0AQ5Tqe2MHuFYvT6tu7xA82EGmU65+P830w@mail.gmail.com>
Subject: Re: OOC on Giraph 1.1 or 1.2
To: dev@giraph.apache.org
Content-Type: multipart/alternative; boundary=001a1137c360250b510526994e8f

--001a1137c360250b510526994e8f
Content-Type: text/plain; charset=UTF-8

Yes, OOC mechanism in 1.1 may not work in certain scenarios:
 - If a worker goes into long GC pauses (which is likely since a job that
is supposed to use OOC feature usually suffer from high memory pressure),
all incoming messages will be buffered before coming to Giraph runtime, and
this will cause he job to crash.
 - The rate at which a worker receives messages may be more than the rate
at which a worker spills messages to disk (specially in applications where
compute function does a little work, but generates a fair number of
messages, such as PageRank). In this case, messages keep accumulating and
the worker crashes.
 - OOC feature is not supported for input superstep for Giraph 1.0 and 1.1.
That being said, if graph size is larger than aggregate available memory,
you are likely to see the job crashes during input superstep.

OOC feature is under active development at this moment. A new OOC mechanism
is recently pushed to the code base (which basically solves the problem for
input superstep and OOC graph). However, the new OOC mechanism still have
problems for OOC messages. An entirely new approach to OOC is being
developed and soon will be available in the code base.

Best,
Hassan

On Thu, Dec 10, 2015 at 8:58 PM, Khaled Ammar <khaled.ammar@gmail.com>
wrote:

> Hi,
>
> I remember that I used OOC on Giraph 1.0, how ever it is not working on
> 1.1.
>
> Is there a known bug in the OOC feature for recent Giraph versions?
>
> --
> Thanks,
> -Khaled
>

--001a1137c360250b510526994e8f--