giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martin Neumann <mneum...@spotify.com>
Subject Re: debugging graph-programs
Date Tue, 18 Feb 2014 16:38:10 GMT
@ Lukas: Do you have some links where I can read how to use it to debug
programs?

My current program has a lot of logic in the EdgeInputFormat as well that
was why I was looking at MapReduce tools since loading the data uses the
same classes (as far as I can see). But I agree its a rather desperate try.

I need a simple way to test the input format, output format and the vertex
program.
For the vertex program I currently develop it on a white bord to get the
interactions right then some unit test to make sure it works on a micro
level. I have not found a good way to test the input format (e.g. if the
Graph was build correctly) or how to do a good end to end test.

cheers Martin

On Tue, Feb 18, 2014 at 5:08 PM, Lukas Nalezenec <
lukas.nalezenec@firma.seznam.cz> wrote:

>  Hi,
> How about org.apache.giraph.utils.TestGraph?
> I would not try using mrunit for testing Giraph.
>
> cheers
> Lukas
>
>
>
> On 18.2.2014 17:01, Mirko Kämpf wrote:
>
>   Hi Martin,
>
>  it depends on the details of your implementation. In principle, you do
> not write Mapper and Reducer
>  classes, which are tested by MR-Unit. You usually have to implement the
> application / algorithm specific
> logic, in e.g. the Vertex class. MR-Unit might not help that much here.
>
>  This leads to the question, if there is a comparable approach / or tool
> like MR Unit for Giraph?
>
>  Best wishes
> Mirko
>
>
>
>
> On Tue, Feb 18, 2014 at 3:52 PM, Martin Neumann <mneumann@spotify.com>wrote:
>
>>  How much of the Giraph 1.1.0 version is still based on MapReduce?
>> Would something like mrunit (http://mrunit.apache.org/) work with it?
>>
>>
>> On Tue, Feb 18, 2014 at 12:52 PM, Martin Neumann <mneumann@spotify.com>wrote:
>>
>>>  I'm using the Yarn version (1.1.0 trunk)  and I was not able to get it
>>> to work that way. But I'm no expert on that, if anyone knows a way to get
>>> it done It would be great.
>>>
>>>
>>> On Tue, Feb 18, 2014 at 2:07 AM, Roman Shaposhnik <roman@shaposhnik.org>wrote:
>>>
>>>> I find running Giraph apps via hadoop's local execution mode
>>>> quite useful for debugging.
>>>>
>>>> Thanks,
>>>> Roman.
>>>>
>>>> On Mon, Feb 17, 2014 at 4:07 AM, Martin Neumann <mneumann@spotify.com>
>>>> wrote:
>>>> > Hej,
>>>> >
>>>> > I find Giraph programs quite difficult to debug (all I have is log
>>>> outputs).
>>>> > Anyone has some hints for debugging tools or coding practices to make
>>>> it
>>>> > simpler?
>>>> >
>>>> > I'm using Giraph 1.1.0 trunk on YARN so I can't use Map/Reduce
>>>> debugging
>>>> > aids (at least I think so, I haven't tried any)
>>>> >
>>>> > cheers
>>>> > Martin
>>>>
>>>
>>>
>>
>
>

Mime
View raw message