Mailing-List: contact giraph-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: giraph-user@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of jghoman@gmail.com designates
 209.85.212.175 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <6EF2673E-4D3F-4BC7-8B33-51117DC88A10@potomacfusion.com>
References: <6EF2673E-4D3F-4BC7-8B33-51117DC88A10@potomacfusion.com>
From: Jakob Homan <jghoman@gmail.com>
Date: Thu, 9 Feb 2012 11:30:22 -0800
Message-ID: 
 <CADiKvVs5ADpVXbkRzZ=+4b5fipCd7mJ0Lc=j4gBo8bHu8GbzQQ@mail.gmail.com>
Subject: Re: Giraph Architecture bug in
To: giraph-user@incubator.apache.org
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

> Since we call waitForCompletion() (which calls submit() internally) in Gi=
raphJob#run(), we cannot override those methods.
Why can't we just call super.waitForCompletion() and override them to
our users?  It seems there are some advantages to GiraphJob being-a
Job (easier integration with schedulers, Pig, etc.), but I'm not sure
if it outweighs the inheritance issues that may be reaised...


On Wed, Feb 8, 2012 at 9:57 PM, David Garcia <dgarcia@potomacfusion.com> wr=
ote:
> Not tonight. =A0But I might be able to tomorrow.
>
> Sent from my HTC Inspire=99 4G on AT&T
>
>
> ----- Reply message -----
> From: "Avery Ching" <aching@apache.org>
> To: "giraph-user@incubator.apache.org" <giraph-user@incubator.apache.org>
> Subject: Giraph Architecture bug in
> Date: Wed, Feb 8, 2012 8:17 pm
>
>
>
> AFAIK we don't have any SOP for opening issues.=A0 Maybe I'll take a crac=
k
> at this one tonight if I find some time, unless you were planning to
> work on it David.
>
> Avery
>
> On 2/8/12 5:46 PM, David Garcia wrote:
>> I opened up
>>
>> * GIRAPH-144<https://issues.apache.org/jira/browse/GIRAPH-144>
>>
>>
>> I apologize if I didn't do it up according to project SOP's.=A0 I haven'=
t
>> had time to read it thoroughly.
>>
>> -David
>>
>>
>> On 2/8/12 7:29 PM, "David Garcia"<dgarcia@potomacfusion.com>=A0 wrote:
>>
>>> Yeah, I'll write something up.
>>>
>>>
>>> On 2/8/12 7:26 PM, "Avery Ching"<aching@apache.org>=A0 wrote:
>>>
>>>> Since we call waitForCompletion() (which calls submit() internally) in
>>>> GiraphJob#run(), we cannot override those methods.=A0 A better fix wou=
ld
>>>> probably be to use composition rather than inheritance (i.e.
>>>>
>>>> public class GiraphJob {
>>>>=A0=A0=A0=A0=A0 Job internalJob;
>>>> }
>>>>
>>>> and expose the methods we would like as necessary.=A0 There are other
>>>> methods we don't want the user to call, (i.e. setMapperClass(), etc.).
>>>> David, can you please open an issue for this?
>>>>
>>>> Avery
>>>>
>>>> On 2/8/12 5:17 PM, David Garcia wrote:
>>>>> This is a very subtle bug.=A0 GiraphJob inherits from
>>>>> org.apache.mapreduce.Job.=A0 However, the methods submit() and
>>>>> waitForCompletion() are not overridden.=A0 I assumed that they were
>>>>> implemented, so when I called either one of these methods, the
>>>>> framework
>>>>> started up identity mappers/reducers.=A0 A simple fix is to throw
>>>>> unsupported operation exceptions or to implement these methods.
>>>>> Perhaps
>>>>> this has been done already?
>>>>>
>>>>> -David
>>>>>
>>>>> On 2/7/12 7:46 PM, "David Garcia"<dgarcia@potomacfusion.com>=A0=A0 wr=
ote:
>>>>>
>>>>>> I am running into a weird error that I haven't seen yet (I suppose
>>>>>> I've
>>>>>> been lucky).=A0 I see the following in the logging:
>>>>>>
>>>>>> org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoo=
p
>>>>>> library for your platform... using builtin-java classes where
>>>>>> applicable
>>>>>>
>>>>>>
>>>>>> In the job definition, the property "mapreduce.map.class" is not eve=
n
>>>>>> defined.=A0 For Giraph, this is usually set to
>>>>>> "mapreduce.map.class=3Dorg.apache.giraph.graph.GraphMapper"
>>>>>>
>>>>>> I'm building my project with hadoop 0.20.204.
>>>>>>
>>>>>> When I build the GiraphProject myself (and run my own tests with the
>>>>>> projects dependencies), I have no problems.=A0 The main difference i=
s
>>>>>> that
>>>>>> I'm using a Giraph dependency in my work project.=A0 All input is
>>>>>> welcome.
>>>>>> Thx!!
>>>>>>
>>>>>> -David
>>>>>>
>