giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roman Shaposhnik <ro...@shaposhnik.org>
Subject Re: debugging graph-programs
Date Tue, 18 Feb 2014 19:14:23 GMT
Well, then you should really use this approach:
     https://git-wip-us.apache.org/repos/asf?p=giraph.git;a=blob;f=giraph-examples/src/test/java/org/apache/giraph/examples/SimpleShortestPathsComputationTest.java;h=dc6c84aeda01283c5d29d2787e89fa01fbe65241;hb=HEAD#l114

Thanks,
Roman.

On Tue, Feb 18, 2014 at 8:38 AM, Martin Neumann <mneumann@spotify.com> wrote:
> @ Lukas: Do you have some links where I can read how to use it to debug
> programs?
>
> My current program has a lot of logic in the EdgeInputFormat as well that
> was why I was looking at MapReduce tools since loading the data uses the
> same classes (as far as I can see). But I agree its a rather desperate try.
>
> I need a simple way to test the input format, output format and the vertex
> program.
> For the vertex program I currently develop it on a white bord to get the
> interactions right then some unit test to make sure it works on a micro
> level. I have not found a good way to test the input format (e.g. if the
> Graph was build correctly) or how to do a good end to end test.
>
> cheers Martin
>
> On Tue, Feb 18, 2014 at 5:08 PM, Lukas Nalezenec
> <lukas.nalezenec@firma.seznam.cz> wrote:
>>
>> Hi,
>> How about org.apache.giraph.utils.TestGraph?
>> I would not try using mrunit for testing Giraph.
>>
>> cheers
>> Lukas
>>
>>
>>
>> On 18.2.2014 17:01, Mirko Kämpf wrote:
>>
>> Hi Martin,
>>
>> it depends on the details of your implementation. In principle, you do not
>> write Mapper and Reducer
>> classes, which are tested by MR-Unit. You usually have to implement the
>> application / algorithm specific
>> logic, in e.g. the Vertex class. MR-Unit might not help that much here.
>>
>> This leads to the question, if there is a comparable approach / or tool
>> like MR Unit for Giraph?
>>
>> Best wishes
>> Mirko
>>
>>
>>
>>
>> On Tue, Feb 18, 2014 at 3:52 PM, Martin Neumann <mneumann@spotify.com>
>> wrote:
>>>
>>> How much of the Giraph 1.1.0 version is still based on MapReduce?
>>> Would something like mrunit (http://mrunit.apache.org/) work with it?
>>>
>>>
>>> On Tue, Feb 18, 2014 at 12:52 PM, Martin Neumann <mneumann@spotify.com>
>>> wrote:
>>>>
>>>> I'm using the Yarn version (1.1.0 trunk)  and I was not able to get it
>>>> to work that way. But I'm no expert on that, if anyone knows a way to get
it
>>>> done It would be great.
>>>>
>>>>
>>>> On Tue, Feb 18, 2014 at 2:07 AM, Roman Shaposhnik <roman@shaposhnik.org>
>>>> wrote:
>>>>>
>>>>> I find running Giraph apps via hadoop's local execution mode
>>>>> quite useful for debugging.
>>>>>
>>>>> Thanks,
>>>>> Roman.
>>>>>
>>>>> On Mon, Feb 17, 2014 at 4:07 AM, Martin Neumann <mneumann@spotify.com>
>>>>> wrote:
>>>>> > Hej,
>>>>> >
>>>>> > I find Giraph programs quite difficult to debug (all I have is log
>>>>> > outputs).
>>>>> > Anyone has some hints for debugging tools or coding practices to
make
>>>>> > it
>>>>> > simpler?
>>>>> >
>>>>> > I'm using Giraph 1.1.0 trunk on YARN so I can't use Map/Reduce
>>>>> > debugging
>>>>> > aids (at least I think so, I haven't tried any)
>>>>> >
>>>>> > cheers
>>>>> > Martin
>>>>
>>>>
>>>
>>
>>
>

Mime
View raw message