Mailing-List: contact user-help@hama.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hama.apache.org
MIME-Version: 1.0
In-Reply-To: <20140224115033.Horde.ncLFV5rV3i213p0mIBKc0w1@webmail.auth.gr>
References: <20140223163757.Horde.ASMviOdWydaih9qDTFSG2g1@webmail.auth.gr>
	<CAGQgZQTVY3OzvZrsxNe+aZm9VpLaTzrS16-fY8hREhENyeHmPw@mail.gmail.com>
	<20140224115033.Horde.ncLFV5rV3i213p0mIBKc0w1@webmail.auth.gr>
Date: Mon, 24 Feb 2014 20:37:05 +0900
Message-ID: 
 <CAGQgZQQ++qr89Na=fqLc=YuJJYamoyvo864COgzrXJyZX0BVGg@mail.gmail.com>
Subject: Re: Aggregator problem
From: "Edward J. Yoon" <edwardyoon@apache.org>
To: "user@hama.apache.org" <user@hama.apache.org>
Content-Type: text/plain; charset=UTF-8

Please see http://svn.apache.org/repos/asf/hama/trunk/examples/src/main/java/org/apache/hama/examples/PageRank.java
and check your program whether it is implemented correctly or not.

I guess, it's not a problem of aggregator. It's a bug in your program.

On Mon, Feb 24, 2014 at 6:50 PM,  <ikapoura@csd.auth.gr> wrote:
> This is the code. What changed and suddenly it throws the error?
>
>
>
> package corever2;
> import java.io.DataInput;
> import java.io.DataOutput;
> import java.io.IOException;
> import org.apache.hadoop.io.DoubleWritable;
> import org.apache.hadoop.io.IntWritable;
> import org.apache.hadoop.io.Text;
> import org.apache.hadoop.io.Writable;
>
> public class MessageCore implements Writable {
>
>     private DoubleWritable type;
>     private Text nodeID;
>     private IntWritable degree;
>     private IntWritable maxDegree;
>     private IntWritable nodeCounter;
>
>     public MessageCore() {
>         type = new DoubleWritable();
>         nodeID = new Text();
>         degree = new IntWritable();
>         maxDegree = new IntWritable();
>         nodeCounter = new IntWritable();
>     }
>
>     public MessageCore(DoubleWritable type, Text nodeID, IntWritable degree,
> IntWritable maxDegree, IntWritable nodeCounter) {
>         this.type = type;
>         this.nodeID = nodeID;
>         this.degree = degree;
>         this.maxDegree = maxDegree;
>         this.nodeCounter = nodeCounter;
>     }
>
>     public MessageCore(DoubleWritable type, Text nodeID) {
>         this.type = type;
>         this.nodeID = nodeID;
>         degree = new IntWritable();
>         maxDegree = new IntWritable();
>         nodeCounter = new IntWritable();
>     }
>
>     public void setNodeCounter(IntWritable nodeCounter) {
>         this.nodeCounter = nodeCounter;
>     }
>
>     public void setMaxDegree(IntWritable maxDegree) {
>         this.maxDegree = maxDegree;
>     }
>
>     public void setType(DoubleWritable type) {
>         this.type = type;
>     }
>
>     public void setNodeID(Text nodeID) {
>         this.nodeID = nodeID;
>     }
>
>     public DoubleWritable getType() {
>         return type;
>     }
>
>     public Text getNodeID() {
>         return nodeID;
>     }
>
>     public IntWritable getDegree() {
>         return degree;
>     }
>
>     public IntWritable getMaxDegree() {
>         return maxDegree;
>     }
>
>     public IntWritable getNodeCounter() {
>         return nodeCounter;
>     }
>
>     public void setDegree(IntWritable degree) {
>         this.degree = degree;
>     }
>
>     @Override
>     public String toString() {
>         return nodeID + "\t" + degree;
>     }
>
>     @Override
>     public void write(DataOutput d) throws IOException {
>         type.write(d);
>         nodeID.write(d);
>         degree.write(d);
>         maxDegree.write(d);
>         nodeCounter.write(d);
>     }
>
>     @Override
>     public void readFields(DataInput di) throws  IOException {
>         type.readFields(di);
>         nodeID.readFields(di);
>         degree.readFields(di);
>         maxDegree.readFields(di);
>         nodeCounter.readFields(di);
>
>     }
> }
>
>
>
> Quoting "Edward J. Yoon" <edwardyoon@apache.org>:
>
>> Please attach your MessageCore.
>>
>> On Sun, Feb 23, 2014 at 11:37 PM,  <ikapoura@csd.auth.gr> wrote:
>>>
>>> Hello there,
>>>
>>> i downloaded hama-trunk a few days ago and till now I am trying to
>>> resolve
>>> the following problem that came up:
>>>
>>> My program runs without errors without aggregators. When I set the
>>> aggregator class to be used, I have the following error (which is like
>>> "cant
>>> find the message class" when the java file exists in the package!):
>>>
>>> corever2 is the package and MessageCore is the name of the class.
>>>
>>> java.io.IOException: can't find class: corever2.MessageCore because
>>> corever2.MessageCore
>>>         at
>>>
>>> org.apache.hadoop.io.AbstractMapWritable.readFields(AbstractMapWritable.java:204)
>>>         at
>>> org.apache.hadoop.io.MapWritable.readFields(MapWritable.java:146)
>>>         at
>>>
>>> org.apache.hama.graph.GraphJobMessage.readFields(GraphJobMessage.java:129)
>>>         at
>>> org.apache.hama.bsp.BSPMessageBundle$1.next(BSPMessageBundle.java:108)
>>>         at
>>> org.apache.hama.bsp.BSPMessageBundle$1.next(BSPMessageBundle.java:78)
>>>         at
>>>
>>> org.apache.hama.bsp.LocalBSPRunner$LocalMessageManager.transfer(LocalBSPRunner.java:356)
>>>         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:381)
>>>         at
>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:133)
>>>         at
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:258)
>>>         at
>>>
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:288)
>>>         at
>>>
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:212)
>>>         at
>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>         at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>>         at
>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>         at
>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>>         at
>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>>         at java.lang.Thread.run(Thread.java:662)
>>> java.io.EOFException
>>>         at java.io.DataInputStream.readFully(DataInputStream.java:180)
>>>         at org.apache.hadoop.io.Text.readFields(Text.java:265)
>>>         at
>>>
>>> org.apache.hama.graph.GraphJobMessage.readFields(GraphJobMessage.java:136)
>>>         at
>>> org.apache.hama.bsp.BSPMessageBundle$1.next(BSPMessageBundle.java:108)
>>>         at
>>> org.apache.hama.bsp.BSPMessageBundle$1.next(BSPMessageBundle.java:78)
>>>         at
>>>
>>> org.apache.hama.bsp.LocalBSPRunner$LocalMessageManager.transfer(LocalBSPRunner.java:356)
>>>         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:381)
>>>         at
>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:133)
>>>         at
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:258)
>>>         at
>>>
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:288)
>>>         at
>>>
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:212)
>>>         at
>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>         at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>>         at
>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>         at
>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>>         at
>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>>         at java.lang.Thread.run(Thread.java:662)
>>> 14/02/23 18:30:23 ERROR bsp.LocalBSPRunner: Exception during BSP
>>> execution!
>>> java.lang.UnsupportedOperationException: Unknown message type:
>>> GraphJobMessage [flag=0, map=null, vertexId=, vertexValue=null]
>>>         at
>>>
>>> org.apache.hama.graph.GraphJobRunner.parseMessages(GraphJobRunner.java:590)
>>>         at
>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:136)
>>>         at
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:258)
>>>         at
>>>
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:288)
>>>         at
>>>
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:212)
>>>         at
>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>         at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>>         at
>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>         at
>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>>         at
>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>>         at java.lang.Thread.run(Thread.java:662)
>>> 14/02/23 18:30:23 ERROR bsp.LocalBSPRunner: Exception during BSP
>>> execution!
>>> java.util.concurrent.ExecutionException:
>>> java.lang.UnsupportedOperationException: Unknown message type:
>>> GraphJobMessage [flag=0, map=null, vertexId=, vertexValue=null]
>>>         at
>>> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>>>         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>>         at
>>>
>>> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:315)
>>>         at java.lang.Thread.run(Thread.java:662)
>>> Caused by: java.lang.UnsupportedOperationException: Unknown message type:
>>> GraphJobMessage [flag=0, map=null, vertexId=, vertexValue=null]
>>>         at
>>>
>>> org.apache.hama.graph.GraphJobRunner.parseMessages(GraphJobRunner.java:590)
>>>         at
>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:136)
>>>         at
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:258)
>>>         at
>>>
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:288)
>>>         at
>>>
>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:212)
>>>         at
>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>         at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>>         at
>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>         at
>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>>         at
>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>>         ... 1 more
>>> 14/02/23 18:30:25 INFO bsp.BSPJobClient: Current supersteps number: 1
>>> 14/02/23 18:30:25 INFO bsp.BSPJobClient: Job failed.
>>>
>>>
>>> The only thing I managed to figure is that this comes up when the
>>> aggregators are enabled.
>>>
>>> Thank you.
>>>
>>
>>
>>
>> --
>> Edward J. Yoon (@eddieyoon)
>> Chief Executive Officer
>> DataSayer, Inc.
>
>
>


-- 
Edward J. Yoon (@eddieyoon)
Chief Executive Officer
DataSayer, Inc.