giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sebastian Schelter <...@apache.org>
Subject Re: Different num supersteps
Date Mon, 03 Mar 2014 07:06:45 GMT
Martin,

can you write a MapReduce job that creates your graph and run it with a 
simpler inputformat?

I really suspect that the bug lies somewhere in your input format.

--sebastian

On 03/02/2014 09:48 PM, Martin Neumann wrote:
> I checked the input just creating the graph and comparing it. While I cant
> say the graph is correct (for its to big) its at least consistent.
>
> So the only things where the different output can come from is the
> connected component part (see code in the first mail). I'm completely
> stomped, the code is basically the example code Giraph ships with all I
> changed was replacing the IntWriteable Id's with Text.
>
> Anyone has any Idea what the problem could be, I'm running out of Idea's.
>
> cheers Martin
>
>
> On Thu, Feb 27, 2014 at 4:59 PM, Sebastian Schelter <ssc@apache.org> wrote:
>
>> Hi Martin
>>
>> I don't think that there are problems with comparing and sorting Text
>> writables as Hadoop is basically a big external sorting system.
>>
>> I'm not sure I understand your edge input reader, it looks very complex,
>> maybe there's a bug somewhere. You could try to preprocess your data using
>> Hadoop so that you can use a simple VertexInputFormat and see if your
>> problems still occur.
>>
>> --sebastian
>>
>>
>> On 02/27/2014 04:41 PM, Martin Neumann wrote:
>>
>>> Hm
>>>
>>> I ran the job 5 times and made a diff between the outputs and they are not
>>> the same. I cant find anything in the code that could lead to this
>>> behaviour.
>>>
>>> The only idea where to look a the moment would be the identifier. Has
>>> anyone experience with String identifier?
>>> Is a possible that there are problems with comparing and sorting
>>> TextWritables?
>>>
>>> cheers Martin
>>>
>>>
>>
>


Mime
View raw message