incubator-giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Avery Ching <ach...@apache.org>
Subject Re: Giraph Architecture bug in
Date Thu, 09 Feb 2012 01:26:10 GMT
Since we call waitForCompletion() (which calls submit() internally) in 
GiraphJob#run(), we cannot override those methods.  A better fix would 
probably be to use composition rather than inheritance (i.e.

public class GiraphJob {
     Job internalJob;
}

and expose the methods we would like as necessary.  There are other 
methods we don't want the user to call, (i.e. setMapperClass(), etc.).  
David, can you please open an issue for this?

Avery

On 2/8/12 5:17 PM, David Garcia wrote:
> This is a very subtle bug.  GiraphJob inherits from
> org.apache.mapreduce.Job.  However, the methods submit() and
> waitForCompletion() are not overridden.  I assumed that they were
> implemented, so when I called either one of these methods, the framework
> started up identity mappers/reducers.  A simple fix is to throw
> unsupported operation exceptions or to implement these methods.  Perhaps
> this has been done already?
>
> -David
>
> On 2/7/12 7:46 PM, "David Garcia"<dgarcia@potomacfusion.com>  wrote:
>
>> I am running into a weird error that I haven't seen yet (I suppose I've
>> been lucky).  I see the following in the logging:
>>
>> org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop
>> library for your platform... using builtin-java classes where applicable
>>
>>
>> In the job definition, the property "mapreduce.map.class" is not even
>> defined.  For Giraph, this is usually set to
>> "mapreduce.map.class=org.apache.giraph.graph.GraphMapper"
>>
>> I'm building my project with hadoop 0.20.204.
>>
>> When I build the GiraphProject myself (and run my own tests with the
>> projects dependencies), I have no problems.  The main difference is that
>> I'm using a Giraph dependency in my work project.  All input is welcome.
>> Thx!!
>>
>> -David
>>


Mime
View raw message