giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jyoti Yadav <rao.jyoti26ya...@gmail.com>
Subject Re: Writing my own aggregator..
Date Sat, 11 Jan 2014 04:10:23 GMT
Hi Ameya..
I am using  Giraph for my class project..


On Fri, Jan 10, 2014 at 10:00 PM, Ameya Vilankar
<ameya.vilankar@gmail.com>wrote:

> Your are welcome. If you don't mind me asking, what are you using Giraph
> for? Class project or on the job?
>
>
> On Fri, Jan 10, 2014 at 4:22 AM, Jyoti Yadav <rao.jyoti26yadav@gmail.com>wrote:
>
>> Thanks a lot Ameya...It really worked..:).
>>
>>
>> On Fri, Jan 10, 2014 at 2:32 PM, Jyoti Yadav <rao.jyoti26yadav@gmail.com>wrote:
>>
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: Ameya Vilankar <ameya.vilankar@gmail.com>
>>> Date: Fri, Jan 10, 2014 at 1:43 PM
>>> Subject: Re: Writing my own aggregator..
>>> To: Jyoti Yadav <rao.jyoti26yadav@gmail.com>
>>>
>>>
>>> This should solve it I think. If it doesn't email me the error.
>>>
>>> // MyArrayWritable.java
>>>
>>> package org.apache.giraph.examples.utils;
>>>
>>> import java.io.*;
>>> import org.apache.hadoop.io.Writable;
>>> import org.apache.hadoop.io.LongWritable;
>>> import org.apache.hadoop.io.WritableComparator;
>>> import java.util.Arrays;
>>> import java.util.*;
>>>
>>> public class MyArrayWritable implements Writable {
>>>
>>>   private ArrayList<Long> arraylist;
>>>
>>>   public MyArrayWritable()
>>>   {
>>>       arraylist = new ArrayList<Long>();
>>>   }
>>>
>>>   public MyArrayWritable(long toAdd)
>>>   {
>>>     arraylist = new ArrayList<Long>();
>>>     arraylist.add(toAdd);
>>>   }
>>>
>>>   public ArrayList<Long> get_arraylist()
>>>   {
>>>     return arraylist;
>>>   }
>>>
>>>   public  void set_arraylist(ArrayList al)
>>>   {
>>>     this.arraylist = al;
>>>   }
>>>
>>>   @Override
>>>   public void readFields(DataInput in) throws IOException {
>>>
>>>     int size = in.readInt();
>>>
>>>     arraylist = new ArrayList<Long>(size);
>>>
>>>     for(int i = 0; i < size; i++)
>>>     {
>>>       arraylist.add(in.readLong());
>>>     }
>>>   }
>>>
>>>   @Override
>>>   public void write(DataOutput out) throws IOException {
>>>
>>>     out.writeInt(arraylist.size());
>>>
>>>     for(int i = 0; i < arraylist.size(); i++)
>>>      {
>>>      out.writeLong(arraylist.get(i));
>>>     }
>>>   }
>>>
>>>   @Override
>>>   public String toString()
>>>   {
>>>         return "output is "+ Long.toString(item) + "\n";
>>>    }
>>> }
>>>
>>>
>>> 2.MyArrayAggregator.java
>>>
>>> package org.apache.giraph.examples.utils;
>>> import org.apache.giraph.aggregators.BasicAggregator;
>>> import java.util.*;
>>>
>>> public class MyArrayAggregator extends BasicAggregator<MyArrayWritable>
{
>>>   @Override
>>>   public void aggregate(MyArrayWritable value) {
>>>      getAggregatedValue().get_arraylist().addAll(value);
>>>   }
>>>
>>>   @Override
>>>   public MyArrayWritable createInitialValue() {
>>>     return new MyArrayWritable();
>>>   }
>>> }
>>>
>>>
>>> On Fri, Jan 10, 2014 at 1:27 AM, Jyoti Yadav <rao.jyoti26yadav@gmail.com
>>> > wrote:
>>>
>>>> Hi Ameya..
>>>>
>>>>  I am badly stuck while implementing my custom aggregator..
>>>> In my program i  want to send each vertex id  to master.
>>>> For that i took an arraylist, in which each vertex is adding its own
>>>> id.while running the program,each vertex calls aggregate() function..As per
>>>> my observation it is working fine in vertex compute method.But while
>>>> retrieving back in master compute function.arraylist is not reflected back
>>>> to master compute function.
>>>>
>>>> I am attaching two files below..You are requested to please check it
>>>> once..
>>>>
>>>> *1.MyArrayWritable.java*
>>>>
>>>> package org.apache.giraph.examples.utils;
>>>>
>>>> import java.io.*;
>>>> import org.apache.hadoop.io.Writable;
>>>> import org.apache.hadoop.io.LongWritable;
>>>> import org.apache.hadoop.io.WritableComparator;
>>>> import java.util.Arrays;
>>>> import java.util.*;
>>>>
>>>>
>>>> public class MyArrayWritable implements Writable {
>>>>
>>>>   private long item;
>>>>   private ArrayList<Long> arraylist=new ArrayList<Long>(5);
>>>>
>>>>   public MyArrayWritable()
>>>>     {
>>>>         item=0;
>>>>
>>>>
>>>>         //arraylist=new ArrayList<Long>(5);
>>>>         arraylist.add(item);
>>>>
>>>>     }
>>>>
>>>>   public MyArrayWritable(long item1)
>>>>     {
>>>>
>>>>     item=item1;
>>>>     //arraylist=new ArrayList<Long>(5);
>>>>     arraylist.add(item);
>>>>
>>>>     }
>>>>
>>>>
>>>>   public ArrayList<Long> get_arraylist() { return arraylist; }
>>>>   public  void set_arraylist(ArrayList al)
>>>>   {
>>>>     //this.arraylist=new ArrayList<Long>(al);
>>>>     this.arraylist=al;
>>>>   }
>>>>   public long get_item(){return item;}
>>>>
>>>>  @Override
>>>>   public void readFields(DataInput in) throws IOException {
>>>>          item=in.readLong();
>>>>     int size=arraylist.size();
>>>>     size=in.readInt();
>>>>
>>>>     arraylist=new ArrayList<Long>(5);
>>>>
>>>>     for(int i=0;i<size;i++)
>>>>     {
>>>>      arraylist.add(in.readLong());
>>>>     }
>>>>   }
>>>>  @Override
>>>>   public void write(DataOutput out) throws IOException {
>>>>           out.writeLong(item);
>>>>
>>>>     out.writeInt(arraylist.size());
>>>>
>>>>
>>>>     for(int i=0;i<arraylist.size();i++)
>>>>     {
>>>>      out.writeLong(arraylist.get(i));
>>>>     }
>>>>
>>>>
>>>>   }
>>>>
>>>> @Override
>>>>   public String toString()
>>>>   {
>>>>         return "output is "+ Long.toString(item) + "\n";
>>>>    }
>>>>
>>>>
>>>>
>>>> }
>>>>
>>>>
>>>> 2.MyArrayAggregator.java
>>>>
>>>> package org.apache.giraph.examples.utils;
>>>> import org.apache.giraph.aggregators.BasicAggregator;
>>>> import java.util.*;
>>>>
>>>>
>>>> public class MyArrayAggregator extends BasicAggregator<MyArrayWritable>
>>>> {
>>>>   @Override
>>>>   public void aggregate(MyArrayWritable value) {
>>>>     ArrayList<Long> al=new ArrayList<Long>();
>>>>     (getAggregatedValue().get_arraylist()).add(value.get_item());
>>>>     al=getAggregatedValue().get_arraylist();
>>>>     getAggregatedValue().set_arraylist(al);
>>>>
>>>>   }
>>>>
>>>>   @Override
>>>>   public MyArrayWritable createInitialValue() {
>>>>     return new MyArrayWritable();
>>>>   }
>>>> }
>>>>
>>>> Thanks in advance ...
>>>>  Jyoti
>>>>
>>>
>>>
>>>
>>
>

Mime
View raw message