incubator-giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Avery Ching <ach...@apache.org>
Subject Re: Giraph Architecture bug in
Date Thu, 09 Feb 2012 02:17:07 GMT
AFAIK we don't have any SOP for opening issues.  Maybe I'll take a crack 
at this one tonight if I find some time, unless you were planning to 
work on it David.

Avery

On 2/8/12 5:46 PM, David Garcia wrote:
> I opened up
>
> * GIRAPH-144<https://issues.apache.org/jira/browse/GIRAPH-144>
>
>
> I apologize if I didn't do it up according to project SOP's.  I haven't
> had time to read it thoroughly.
>
> -David
>
>
> On 2/8/12 7:29 PM, "David Garcia"<dgarcia@potomacfusion.com>  wrote:
>
>> Yeah, I'll write something up.
>>
>>
>> On 2/8/12 7:26 PM, "Avery Ching"<aching@apache.org>  wrote:
>>
>>> Since we call waitForCompletion() (which calls submit() internally) in
>>> GiraphJob#run(), we cannot override those methods.  A better fix would
>>> probably be to use composition rather than inheritance (i.e.
>>>
>>> public class GiraphJob {
>>>      Job internalJob;
>>> }
>>>
>>> and expose the methods we would like as necessary.  There are other
>>> methods we don't want the user to call, (i.e. setMapperClass(), etc.).
>>> David, can you please open an issue for this?
>>>
>>> Avery
>>>
>>> On 2/8/12 5:17 PM, David Garcia wrote:
>>>> This is a very subtle bug.  GiraphJob inherits from
>>>> org.apache.mapreduce.Job.  However, the methods submit() and
>>>> waitForCompletion() are not overridden.  I assumed that they were
>>>> implemented, so when I called either one of these methods, the
>>>> framework
>>>> started up identity mappers/reducers.  A simple fix is to throw
>>>> unsupported operation exceptions or to implement these methods.
>>>> Perhaps
>>>> this has been done already?
>>>>
>>>> -David
>>>>
>>>> On 2/7/12 7:46 PM, "David Garcia"<dgarcia@potomacfusion.com>   wrote:
>>>>
>>>>> I am running into a weird error that I haven't seen yet (I suppose
>>>>> I've
>>>>> been lucky).  I see the following in the logging:
>>>>>
>>>>> org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop
>>>>> library for your platform... using builtin-java classes where
>>>>> applicable
>>>>>
>>>>>
>>>>> In the job definition, the property "mapreduce.map.class" is not even
>>>>> defined.  For Giraph, this is usually set to
>>>>> "mapreduce.map.class=org.apache.giraph.graph.GraphMapper"
>>>>>
>>>>> I'm building my project with hadoop 0.20.204.
>>>>>
>>>>> When I build the GiraphProject myself (and run my own tests with the
>>>>> projects dependencies), I have no problems.  The main difference is
>>>>> that
>>>>> I'm using a Giraph dependency in my work project.  All input is
>>>>> welcome.
>>>>> Thx!!
>>>>>
>>>>> -David
>>>>>


Mime
View raw message