giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mohammed_fa...@thbs.com
Subject Re: Writing my own aggregator..
Date Mon, 13 Jan 2014 05:39:34 GMT

HI , I will be using Giraph to produce a poc at work and i am very new to hadoop . Please
provide with suggestion on how to go about learning giraph
 
-----Original Message-----
From: "Ameya Vilankar" <ameya.vilankar@gmail.com>
Sent: Friday, January 10, 2014 10:00pm
To: user@giraph.apache.org
Subject: Re: Writing my own aggregator..



Your are welcome. If you don't mind me asking, what are you using Giraph for? Class project
or on the job?



On Fri, Jan 10, 2014 at 4:22 AM, Jyoti Yadav <[mailto:rao.jyoti26yadav@gmail.com] rao.jyoti26yadav@gmail.com>
wrote:

Thanks a lot Ameya...It really worked..:).





On Fri, Jan 10, 2014 at 2:32 PM, Jyoti Yadav <[mailto:rao.jyoti26yadav@gmail.com] rao.jyoti26yadav@gmail.com>
wrote:





---------- Forwarded message ----------
From: Ameya Vilankar <[mailto:ameya.vilankar@gmail.com] ameya..vilankar@gmail.com>
 Date: Fri, Jan 10, 2014 at 1:43 PM
Subject: Re: Writing my own aggregator..
To: Jyoti Yadav <[mailto:rao.jyoti26yadav@gmail.com] rao.jyoti26yadav@gmail.com>




This should solve it I think. If it doesn't email me the error.



// MyArrayWritable.java

package org.apache.giraph.examples.utils;
import java.io.*;
import org.apache.hadoop.io.Writable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.WritableComparator;
import java.util.Arrays;
import java.util.*;
public class MyArrayWritable implements Writable {
private ArrayList<Long> arraylist;

public MyArrayWritable()
{
arraylist = new ArrayList<Long>();
}
public MyArrayWritable(long toAdd)
{
arraylist = new ArrayList<Long>();
arraylist.add(toAdd);

}

public ArrayList<Long> get_arraylist()
{
return arraylist;
}

public  void set_arraylist(ArrayList al)
{
this.arraylist = al;

}
@Override
public void readFields(DataInput in) throws IOException {

int size = in.readInt();

arraylist = new ArrayList<Long>(size);
for(int i = 0; i < size; i++)

{
arraylist.add(in.readLong());
}
}
@Override
public void write(DataOutput out) throws IOException {

out.writeInt(arraylist.size());

for(int i = 0; i < arraylist.size(); i++)

{
out.writeLong(arraylist.get(i));
}
}
@Override
public String toString()
{
return "output is "+ Long.toString(item) + "\n";
}
}

2.MyArrayAggregator.java
package org.apache.giraph.examples.utils;
import org.apache.giraph.aggregators.BasicAggregator;
import java.util.*;
public class MyArrayAggregator extends BasicAggregator<MyArrayWritable> {
@Override
public void aggregate(MyArrayWritable value) {
getAggregatedValue().get_arraylist().addAll(value);

}
@Override
public MyArrayWritable createInitialValue() {
return new MyArrayWritable();
}
}







On Fri, Jan 10, 2014 at 1:27 AM, Jyoti Yadav <[mailto:rao.jyoti26yadav@gmail.com] rao.jyoti26yadav@gmail.com>
wrote:






Hi Ameya..

I am badly stuck while implementing my custom aggregator..

In my program i  want to send each vertex id  to master.

For that i took an arraylist, in which each vertex is adding its own id.while running the
program,each vertex calls aggregate() function..As per my observation it is working fine in
vertex compute method.But while retrieving back in master compute function.arraylist is not
reflected back to master compute function.



I am attaching two files below..You are requested to please check it once..



1.MyArrayWritable.java


package org.apache.giraph.examples.utils;

import java.io.*;
 import org.apache.hadoop.io.Writable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.WritableComparator;
import java.util.Arrays;
import java.util.*;


public class MyArrayWritable implements Writable {

 private long item;
 private ArrayList<Long> arraylist=new ArrayList<Long>(5);
 
 public MyArrayWritable()
 {
 item=0;
 
 
 //arraylist=new ArrayList<Long>(5);
 arraylist.add(item);
 
 }

 public MyArrayWritable(long item1) 
 { 
 
 item=item1;
 //arraylist=new ArrayList<Long>(5);
 arraylist.add(item);
 
 }
 
 
 public ArrayList<Long> get_arraylist() { return arraylist; }
 public  void set_arraylist(ArrayList al)
 {
 //this.arraylist=new ArrayList<Long>(al);
 this.arraylist=al;
 }
 public long get_item(){return item;}
 
 @Override
 public void readFields(DataInput in) throws IOException {
 item=in.readLong();
 int size=arraylist.size();
 size=in.readInt();
 
 arraylist=new ArrayList<Long>(5);

 for(int i=0;i<size;i++)
 {
 arraylist.add(in.readLong());
 }
 }
 @Override
 public void write(DataOutput out) throws IOException {
 out.writeLong(item);
 
 out.writeInt(arraylist.size());
 

 for(int i=0;i<arraylist.size();i++)
 {
 out.writeLong(arraylist.get(i));
 }    
 
 
 }

@Override
 public String toString() 
 {
 return "output is "+ Long.toString(item) + "\n";
 }

 
 
}
  


2.MyArrayAggregator.java


package org.apache.giraph.examples.utils;
import org.apache.giraph.aggregators.BasicAggregator;
import java.util.*;


public class MyArrayAggregator extends BasicAggregator<MyArrayWritable> {
 @Override
 public void aggregate(MyArrayWritable value) {
 ArrayList<Long> al=new ArrayList<Long>();
 (getAggregatedValue().get_arraylist()).add(value.get_item());
 al=getAggregatedValue().get_arraylist();
 getAggregatedValue().set_arraylist(al);
 
 }

 @Override
 public MyArrayWritable createInitialValue() {
 return new MyArrayWritable();
 }
}


Thanks in advance ...

Jyoti


******* DISCLAIMER: This email and any files transmitted with it are privileged and confidential
information and intended solely for the use of the individual or entity to which they are
addressed. If you are not the intended recipient, please notify the sender by e-mail and delete
the original message. Further, you are not to copy, disclose, or distribute this e-mail or
its contents to any other person and any such actions are unlawful. This e-mail may contain
viruses. Torry Harris Business Solutions has taken every reasonable precaution to minimize
this risk, but is not liable for any damage you may sustain as a result of any virus in this
e-mail.  The recipient should check this email and any attachments for the presence of viruses.
THBS reserves the right to monitor and review the content of all messages sent to or from
this e-mail address********



******* DISCLAIMER: This email and any files transmitted with it are privileged and confidential
information and intended solely for the use of the individual or entity to which they are
addressed. If you are not the intended recipient, please notify the sender by e-mail and delete
the original message. Further, you are not to copy, disclose, or distribute this e-mail or
its contents to any other person and any such actions are unlawful. This e-mail may contain
viruses. Torry Harris Business Solutions has taken every reasonable precaution to minimize
this risk, but is not liable for any damage you may sustain as a result of any virus in this
e-mail.  The recipient should check this email and any attachments for the presence of viruses.
THBS reserves the right to monitor and review the content of all messages sent to or from
this e-mail address.********

Mime
View raw message