incubator-giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Avery Ching <ach...@apache.org>
Subject Re: Unable to load vertices
Date Tue, 27 Dec 2011 17:14:31 GMT
Glad you found the issue Sebastian! =)

Avery

On 12/27/11 12:47 PM, Sebastian Schelter wrote:
> You were write it was an issue with writing/reading the vertex value.
> Only took me three days of searching to find out that I simply forgot to
> call setVertexValue() ... :)
>
> --sebastian
>
>
>
> On 23.12.2011 18:28, Avery Ching wrote:
>> Without looking at your code, maybe your I, V, E, or M types might have
>> Writable issues?  In the single worker case, does checkpointing work?
>> That would verify the writing part of Writable is okay, but not the
>> reading part...(well you can do a manual checkpoint restart I guess to
>> verify that).
>>
>> Avery
>>
>> On 12/23/11 9:23 AM, Sebastian Schelter wrote:
>>> I'm extending org.apache.giraph.graph.Vertex directly. I also created
>>> unit tests for the serialization of the Writables (writing them to a
>>> byte array and reading them back) without finding something. Thank you
>>> for the advice however, I'll continue searching :)
>>>
>>> --sebastian
>>>
>>>
>>> On 23.12.2011 18:14, Avery Ching wrote:
>>>> What MutableVertex implementation are you using?  Sounds like the issue
>>>> only happens during the RPC to send the vertex to another worker.  Maybe
>>>> a bug in the Writable implementation?
>>>>
>>>> Avery
>>>>
>>>> On 12/23/11 3:14 AM, Sebastian Schelter wrote:
>>>>> Hmm, the job works if I use a single worker only locally, strange...
>>>>>
>>>>> On 23.12.2011 11:07, Claudio Martella wrote:
>>>>>> With a super quick look, so i might be completely wrong, this looks
>>>>>> like you're running a different hadoop locally and on your test.
Is
>>>>>> there any chance you're not using hadoop non_secure locally but you're
>>>>>> in your distributed mode?
>>>>>>
>>>>>> On Fri, Dec 23, 2011 at 10:49 AM, Sebastian Schelter<ssc@apache.org>
>>>>>> wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> I'm currently implementing an algorithm for diameter and radius
>>>>>>> estimation. It already works when I run it on toy data via
>>>>>>> InternalVertexRunner in a unit test.
>>>>>>>
>>>>>>> Unfortunately, in my tests with a single node hadoop instance
and
>>>>>>> real
>>>>>>> cluster, I always run into the attached exception during startup.
>>>>>>> Does
>>>>>>> anybody have an idea what might cause this?
>>>>>>>
>>>>>>> --sebastian
>>>>>>>
>>>>>>>
>>>>>>> 2011-12-23 10:43:09,769 INFO org.apache.hadoop.mapred.TaskInProgress:
>>>>>>> Error from attempt_201112230924_0006_m_000001_0:
>>>>>>> java.lang.IllegalStateException: run: Caught an unrecoverable
>>>>>>> exception
>>>>>>> setup: Offlining servers due to exception...
>>>>>>>           at
>>>>>>> org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:641)
>>>>>>>           at
>>>>>>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
>>>>>>>           at org.apache.hadoop.mapred.MapTask.run(MapTask.java:369)
>>>>>>>           at org.apache.hadoop.mapred.Child$4.run(Child.java:259)
>>>>>>>           at java.security.AccessController.doPrivileged(Native
>>>>>>> Method)
>>>>>>>           at javax.security.auth.Subject.doAs(Subject.java:396)
>>>>>>>           at
>>>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>>>>>>>
>>>>>>>
>>>>>>>           at org.apache.hadoop.mapred.Child.main(Child.java:253)
>>>>>>> Caused by: java.lang.RuntimeException: setup: Offlining servers
>>>>>>> due to
>>>>>>> exception...
>>>>>>>           at
>>>>>>> org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:466)
>>>>>>>           at
>>>>>>> org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:630)
>>>>>>>           ... 7 more
>>>>>>> Caused by: java.lang.IllegalStateException: setup: loadVertices
>>>>>>> failed
>>>>>>>           at
>>>>>>> org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:582)
>>>>>>>
>>>>>>>
>>>>>>>           at
>>>>>>> org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:458)
>>>>>>>           ... 8 more
>>>>>>> Caused by: java.lang.RuntimeException: java.io.IOException: Call
to
>>>>>>> poodle-6/127.0.1.1:30002 failed on local exception:
>>>>>>> java.io.EOFException
>>>>>>>           at
>>>>>>> org.apache.giraph.comm.BasicRPCCommunications.sendPartitionReq(BasicRPCCommunications.java:768)
>>>>>>>
>>>>>>>
>>>>>>>           at
>>>>>>> org.apache.giraph.graph.BspServiceWorker.loadVertices(BspServiceWorker.java:304)
>>>>>>>
>>>>>>>
>>>>>>>           at
>>>>>>> org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:575)
>>>>>>>
>>>>>>>
>>>>>>>           ... 9 more
>>>>>>> Caused by: java.io.IOException: Call to poodle-6/127.0.1.1:30002
>>>>>>> failed
>>>>>>> on local exception: java.io.EOFException
>>>>>>>           at
>>>>>>> org.apache.hadoop.ipc.Client.wrapException(Client.java:1065)
>>>>>>>           at org.apache.hadoop.ipc.Client.call(Client.java:1033)
>>>>>>>           at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
>>>>>>>           at $Proxy3.putVertexList(Unknown Source)
>>>>>>>           at
>>>>>>> org.apache.giraph.comm.BasicRPCCommunications.sendPartitionReq(BasicRPCCommunications.java:765)
>>>>>>>
>>>>>>>
>>>>>>>           ... 11 more
>>>>>>> Caused by: java.io.EOFException
>>>>>>>           at java.io.DataInputStream.readInt(DataInputStream.java:375)
>>>>>>>           at
>>>>>>> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:767)
>>>>>>>
>>>>>>>
>>>>>>>           at
>>>>>>> org.apache.hadoop.ipc.Client$Connection.run(Client.java:712)


Mime
View raw message