giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vernon Thommeret <syno...@gmail.com>
Subject Re: Getting SimpleTriangleClosingVertex to run
Date Sun, 07 Oct 2012 19:11:49 GMT
Thanks. I ended up getting it working. Having some issues scaling it,
but working on it.

On Mon, Sep 24, 2012 at 1:17 PM, Eli Reisman <apache.mailbox@gmail.com> wrote:
> The io format types have to be compatible. Since
> IdWithValueVertexOutputFormat does not specify the types it takes, it just
> attempts to output them as using the Writable interface, I use it to output
> data from the SimpleTriangleClosingVertex. I also had to write an
> InputFormat to accept IntWritable id's and IntWritable out-edge
> destinations. Otherwise, should work.
>
>
>
> On Mon, Sep 24, 2012 at 12:06 AM, Avery Ching <aching@apache.org> wrote:
>>
>> I don't think the types are compatible.
>>
>> public class SimpleTriangleClosingVertex extends EdgeListVertex<
>>   IntWritable, SimpleTriangleClosingVertex.IntArrayListWritable,
>>   NullWritable, IntWritable>
>>
>> You'll need to use an input format and output format that fits these
>> types.  Otherwise the issue is likely to be serialization/deserialization
>> here.
>>
>>
>> On 9/23/12 10:44 PM, Vernon Thommeret wrote:
>>>
>>> I'm trying to get the SimpleTriangleClosingVertex to run, but getting
>>> this error:
>>>
>>> java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException: IPC
>>> server unable to read call parameters: null
>>>         at
>>> org.apache.giraph.comm.BasicRPCCommunications.sendPartitionRequest(BasicRPCCommunications.java:923)
>>>         at
>>> org.apache.giraph.graph.BspServiceWorker.loadVertices(BspServiceWorker.java:327)
>>>         at
>>> org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:604)
>>>         at
>>> org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:377)
>>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:578)
>>>         at
>>> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
>>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
>>>         at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>         at javax.security.auth.Subject.doAs(Subject.java:396)
>>>         at
>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
>>>         at org.apache.hadoop.mapred.Child.main(Child.java:264)
>>> Caused by: org.apache.hadoop.ipc.RemoteException: IPC server
>>>
>>> This is the diff that causes the issue:
>>>
>>> @@ -33,7 +33,7 @@ import org.apache.hadoop.fs.Path;
>>>   import org.apache.hadoop.io.IntWritable;
>>>
>>>   import org.apache.giraph.graph.GiraphJob;
>>> -import org.apache.giraph.graph.IntIntNullIntVertex;
>>> +import org.apache.giraph.examples.SimpleTriangleClosingVertex;
>>>   import org.apache.giraph.io.IntIntNullIntTextInputFormat;
>>>   import org.apache.giraph.io.AdjacencyListTextVertexOutputFormat;
>>>
>>> @@ -44,16 +44,12 @@ import org.apache.log4j.Logger;
>>>   /**
>>>    * Simple function to return the in degree for each vertex.
>>>    */
>>> -public class SharedConnectionsVertex extends IntIntNullIntVertex
>>> implements Tool {
>>> +public class SharedConnections implements Tool {
>>>
>>>     private Configuration conf;
>>>     private static final Logger LOG =
>>>         Logger.getLogger(SharedConnections.class);
>>>
>>> -  public void compute(Iterable<IntWritable> messages) {
>>> -    voteToHalt();
>>> -  }
>>> -
>>>     @Override
>>>     public final int run(final String[] args) throws Exception {
>>>       Options options = new Options();
>>> @@ -71,7 +67,7 @@ public class SharedConnections extends
>>> IntIntNullIntVertex implements Tool {
>>>
>>>       GiraphJob job = new GiraphJob(getConf(), getClass().getName());
>>>
>>> -    job.setVertexClass(SharedConnections.class);
>>> +    job.setVertexClass(SimpleTriangleClosingVertex.class);
>>>       job.setVertexInputFormatClass(IntIntNullIntTextInputFormat.class);
>>>
>>> job.setVertexOutputFormatClass(AdjacencyListTextVertexOutputFormat.class);
>>>       job.setWorkerConfiguration(10, 10, 100.0f);
>>>
>>> --
>>>
>>> I.e. I have a dummy job that just outputs the vertices which works,
>>> but trying to switch the vertex class doesn't seem to work. I'm
>>> running the latest version of Giraph (rev 1388628). Should this work
>>> or should I try something different?
>>>
>>> Thanks!
>>> Vernon
>>
>>
>

Mime
View raw message