giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ameya Vilankar <ameya.vilan...@gmail.com>
Subject Re: Writing my own aggregator..
Date Fri, 10 Jan 2014 16:30:04 GMT
Your are welcome. If you don't mind me asking, what are you using Giraph
for? Class project or on the job?


On Fri, Jan 10, 2014 at 4:22 AM, Jyoti Yadav <rao.jyoti26yadav@gmail.com>wrote:

> Thanks a lot Ameya...It really worked..:).
>
>
> On Fri, Jan 10, 2014 at 2:32 PM, Jyoti Yadav <rao.jyoti26yadav@gmail.com>wrote:
>
>>
>>
>> ---------- Forwarded message ----------
>> From: Ameya Vilankar <ameya.vilankar@gmail.com>
>> Date: Fri, Jan 10, 2014 at 1:43 PM
>> Subject: Re: Writing my own aggregator..
>> To: Jyoti Yadav <rao.jyoti26yadav@gmail.com>
>>
>>
>> This should solve it I think. If it doesn't email me the error.
>>
>> // MyArrayWritable.java
>>
>> package org.apache.giraph.examples.utils;
>>
>> import java.io.*;
>> import org.apache.hadoop.io.Writable;
>> import org.apache.hadoop.io.LongWritable;
>> import org.apache.hadoop.io.WritableComparator;
>> import java.util.Arrays;
>> import java.util.*;
>>
>> public class MyArrayWritable implements Writable {
>>
>>   private ArrayList<Long> arraylist;
>>
>>   public MyArrayWritable()
>>   {
>>       arraylist = new ArrayList<Long>();
>>   }
>>
>>   public MyArrayWritable(long toAdd)
>>   {
>>     arraylist = new ArrayList<Long>();
>>     arraylist.add(toAdd);
>>   }
>>
>>   public ArrayList<Long> get_arraylist()
>>   {
>>     return arraylist;
>>   }
>>
>>   public  void set_arraylist(ArrayList al)
>>   {
>>     this.arraylist = al;
>>   }
>>
>>   @Override
>>   public void readFields(DataInput in) throws IOException {
>>
>>     int size = in.readInt();
>>
>>     arraylist = new ArrayList<Long>(size);
>>
>>     for(int i = 0; i < size; i++)
>>     {
>>       arraylist.add(in.readLong());
>>     }
>>   }
>>
>>   @Override
>>   public void write(DataOutput out) throws IOException {
>>
>>     out.writeInt(arraylist.size());
>>
>>     for(int i = 0; i < arraylist.size(); i++)
>>      {
>>      out.writeLong(arraylist.get(i));
>>     }
>>   }
>>
>>   @Override
>>   public String toString()
>>   {
>>         return "output is "+ Long.toString(item) + "\n";
>>    }
>> }
>>
>>
>> 2.MyArrayAggregator.java
>>
>> package org.apache.giraph.examples.utils;
>> import org.apache.giraph.aggregators.BasicAggregator;
>> import java.util.*;
>>
>> public class MyArrayAggregator extends BasicAggregator<MyArrayWritable> {
>>   @Override
>>   public void aggregate(MyArrayWritable value) {
>>      getAggregatedValue().get_arraylist().addAll(value);
>>   }
>>
>>   @Override
>>   public MyArrayWritable createInitialValue() {
>>     return new MyArrayWritable();
>>   }
>> }
>>
>>
>> On Fri, Jan 10, 2014 at 1:27 AM, Jyoti Yadav <rao.jyoti26yadav@gmail.com>wrote:
>>
>>> Hi Ameya..
>>>
>>>  I am badly stuck while implementing my custom aggregator..
>>> In my program i  want to send each vertex id  to master.
>>> For that i took an arraylist, in which each vertex is adding its own
>>> id.while running the program,each vertex calls aggregate() function..As per
>>> my observation it is working fine in vertex compute method.But while
>>> retrieving back in master compute function.arraylist is not reflected back
>>> to master compute function.
>>>
>>> I am attaching two files below..You are requested to please check it
>>> once..
>>>
>>> *1.MyArrayWritable.java*
>>>
>>> package org.apache.giraph.examples.utils;
>>>
>>> import java.io.*;
>>> import org.apache.hadoop.io.Writable;
>>> import org.apache.hadoop.io.LongWritable;
>>> import org.apache.hadoop.io.WritableComparator;
>>> import java.util.Arrays;
>>> import java.util.*;
>>>
>>>
>>> public class MyArrayWritable implements Writable {
>>>
>>>   private long item;
>>>   private ArrayList<Long> arraylist=new ArrayList<Long>(5);
>>>
>>>   public MyArrayWritable()
>>>     {
>>>         item=0;
>>>
>>>
>>>         //arraylist=new ArrayList<Long>(5);
>>>         arraylist.add(item);
>>>
>>>     }
>>>
>>>   public MyArrayWritable(long item1)
>>>     {
>>>
>>>     item=item1;
>>>     //arraylist=new ArrayList<Long>(5);
>>>     arraylist.add(item);
>>>
>>>     }
>>>
>>>
>>>   public ArrayList<Long> get_arraylist() { return arraylist; }
>>>   public  void set_arraylist(ArrayList al)
>>>   {
>>>     //this.arraylist=new ArrayList<Long>(al);
>>>     this.arraylist=al;
>>>   }
>>>   public long get_item(){return item;}
>>>
>>>  @Override
>>>   public void readFields(DataInput in) throws IOException {
>>>          item=in.readLong();
>>>     int size=arraylist.size();
>>>     size=in.readInt();
>>>
>>>     arraylist=new ArrayList<Long>(5);
>>>
>>>     for(int i=0;i<size;i++)
>>>     {
>>>      arraylist.add(in.readLong());
>>>     }
>>>   }
>>>  @Override
>>>   public void write(DataOutput out) throws IOException {
>>>           out.writeLong(item);
>>>
>>>     out.writeInt(arraylist.size());
>>>
>>>
>>>     for(int i=0;i<arraylist.size();i++)
>>>     {
>>>      out.writeLong(arraylist.get(i));
>>>     }
>>>
>>>
>>>   }
>>>
>>> @Override
>>>   public String toString()
>>>   {
>>>         return "output is "+ Long.toString(item) + "\n";
>>>    }
>>>
>>>
>>>
>>> }
>>>
>>>
>>> 2.MyArrayAggregator.java
>>>
>>> package org.apache.giraph.examples.utils;
>>> import org.apache.giraph.aggregators.BasicAggregator;
>>> import java.util.*;
>>>
>>>
>>> public class MyArrayAggregator extends BasicAggregator<MyArrayWritable>
{
>>>   @Override
>>>   public void aggregate(MyArrayWritable value) {
>>>     ArrayList<Long> al=new ArrayList<Long>();
>>>     (getAggregatedValue().get_arraylist()).add(value.get_item());
>>>     al=getAggregatedValue().get_arraylist();
>>>     getAggregatedValue().set_arraylist(al);
>>>
>>>   }
>>>
>>>   @Override
>>>   public MyArrayWritable createInitialValue() {
>>>     return new MyArrayWritable();
>>>   }
>>> }
>>>
>>> Thanks in advance ...
>>>  Jyoti
>>>
>>
>>
>>
>

Mime
View raw message