giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Garcia <>
Subject Re: Giraph Architecture bug in
Date Thu, 09 Feb 2012 01:29:32 GMT
Yeah, I'll write something up.

On 2/8/12 7:26 PM, "Avery Ching" <> wrote:

>Since we call waitForCompletion() (which calls submit() internally) in
>GiraphJob#run(), we cannot override those methods.  A better fix would
>probably be to use composition rather than inheritance (i.e.
>public class GiraphJob {
>     Job internalJob;
>and expose the methods we would like as necessary.  There are other
>methods we don't want the user to call, (i.e. setMapperClass(), etc.).
>David, can you please open an issue for this?
>On 2/8/12 5:17 PM, David Garcia wrote:
>> This is a very subtle bug.  GiraphJob inherits from
>> org.apache.mapreduce.Job.  However, the methods submit() and
>> waitForCompletion() are not overridden.  I assumed that they were
>> implemented, so when I called either one of these methods, the framework
>> started up identity mappers/reducers.  A simple fix is to throw
>> unsupported operation exceptions or to implement these methods.  Perhaps
>> this has been done already?
>> -David
>> On 2/7/12 7:46 PM, "David Garcia"<>  wrote:
>>> I am running into a weird error that I haven't seen yet (I suppose I've
>>> been lucky).  I see the following in the logging:
>>> org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop
>>> library for your platform... using builtin-java classes where
>>> In the job definition, the property "" is not even
>>> defined.  For Giraph, this is usually set to
>>> ""
>>> I'm building my project with hadoop 0.20.204.
>>> When I build the GiraphProject myself (and run my own tests with the
>>> projects dependencies), I have no problems.  The main difference is
>>> I'm using a Giraph dependency in my work project.  All input is
>>> Thx!!
>>> -David

View raw message