incubator-giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Avery Ching <ach...@apache.org>
Subject Re: Unable to load vertices
Date Fri, 23 Dec 2011 17:28:22 GMT
Without looking at your code, maybe your I, V, E, or M types might have 
Writable issues?  In the single worker case, does checkpointing work?  
That would verify the writing part of Writable is okay, but not the 
reading part...(well you can do a manual checkpoint restart I guess to 
verify that).

Avery

On 12/23/11 9:23 AM, Sebastian Schelter wrote:
> I'm extending org.apache.giraph.graph.Vertex directly. I also created
> unit tests for the serialization of the Writables (writing them to a
> byte array and reading them back) without finding something. Thank you
> for the advice however, I'll continue searching :)
>
> --sebastian
>
>
> On 23.12.2011 18:14, Avery Ching wrote:
>> What MutableVertex implementation are you using?  Sounds like the issue
>> only happens during the RPC to send the vertex to another worker.  Maybe
>> a bug in the Writable implementation?
>>
>> Avery
>>
>> On 12/23/11 3:14 AM, Sebastian Schelter wrote:
>>> Hmm, the job works if I use a single worker only locally, strange...
>>>
>>> On 23.12.2011 11:07, Claudio Martella wrote:
>>>> With a super quick look, so i might be completely wrong, this looks
>>>> like you're running a different hadoop locally and on your test. Is
>>>> there any chance you're not using hadoop non_secure locally but you're
>>>> in your distributed mode?
>>>>
>>>> On Fri, Dec 23, 2011 at 10:49 AM, Sebastian Schelter<ssc@apache.org>
>>>> wrote:
>>>>> Hi,
>>>>>
>>>>> I'm currently implementing an algorithm for diameter and radius
>>>>> estimation. It already works when I run it on toy data via
>>>>> InternalVertexRunner in a unit test.
>>>>>
>>>>> Unfortunately, in my tests with a single node hadoop instance and real
>>>>> cluster, I always run into the attached exception during startup. Does
>>>>> anybody have an idea what might cause this?
>>>>>
>>>>> --sebastian
>>>>>
>>>>>
>>>>> 2011-12-23 10:43:09,769 INFO org.apache.hadoop.mapred.TaskInProgress:
>>>>> Error from attempt_201112230924_0006_m_000001_0:
>>>>> java.lang.IllegalStateException: run: Caught an unrecoverable exception
>>>>> setup: Offlining servers due to exception...
>>>>>          at
>>>>> org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:641)
>>>>>          at
>>>>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
>>>>>          at org.apache.hadoop.mapred.MapTask.run(MapTask.java:369)
>>>>>          at org.apache.hadoop.mapred.Child$4.run(Child.java:259)
>>>>>          at java.security.AccessController.doPrivileged(Native Method)
>>>>>          at javax.security.auth.Subject.doAs(Subject.java:396)
>>>>>          at
>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>>>>>
>>>>>          at org.apache.hadoop.mapred.Child.main(Child.java:253)
>>>>> Caused by: java.lang.RuntimeException: setup: Offlining servers due to
>>>>> exception...
>>>>>          at
>>>>> org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:466)
>>>>>          at
>>>>> org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:630)
>>>>>          ... 7 more
>>>>> Caused by: java.lang.IllegalStateException: setup: loadVertices failed
>>>>>          at
>>>>> org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:582)
>>>>>
>>>>>          at
>>>>> org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:458)
>>>>>          ... 8 more
>>>>> Caused by: java.lang.RuntimeException: java.io.IOException: Call to
>>>>> poodle-6/127.0.1.1:30002 failed on local exception:
>>>>> java.io.EOFException
>>>>>          at
>>>>> org.apache.giraph.comm.BasicRPCCommunications.sendPartitionReq(BasicRPCCommunications.java:768)
>>>>>
>>>>>          at
>>>>> org.apache.giraph.graph.BspServiceWorker.loadVertices(BspServiceWorker.java:304)
>>>>>
>>>>>          at
>>>>> org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:575)
>>>>>
>>>>>          ... 9 more
>>>>> Caused by: java.io.IOException: Call to poodle-6/127.0.1.1:30002 failed
>>>>> on local exception: java.io.EOFException
>>>>>          at org.apache.hadoop.ipc.Client.wrapException(Client.java:1065)
>>>>>          at org.apache.hadoop.ipc.Client.call(Client.java:1033)
>>>>>          at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
>>>>>          at $Proxy3.putVertexList(Unknown Source)
>>>>>          at
>>>>> org.apache.giraph.comm.BasicRPCCommunications.sendPartitionReq(BasicRPCCommunications.java:765)
>>>>>
>>>>>          ... 11 more
>>>>> Caused by: java.io.EOFException
>>>>>          at java.io.DataInputStream.readInt(DataInputStream.java:375)
>>>>>          at
>>>>> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:767)
>>>>>
>>>>>          at org.apache.hadoop.ipc.Client$Connection.run(Client.java:712)
>>>>


Mime
View raw message