Return-Path: X-Original-To: apmail-hama-user-archive@www.apache.org Delivered-To: apmail-hama-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8A24F18569 for ; Thu, 28 Jan 2016 11:33:23 +0000 (UTC) Received: (qmail 43845 invoked by uid 500); 28 Jan 2016 11:33:10 -0000 Delivered-To: apmail-hama-user-archive@hama.apache.org Received: (qmail 43817 invoked by uid 500); 28 Jan 2016 11:33:10 -0000 Mailing-List: contact user-help@hama.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hama.apache.org Delivered-To: mailing list user@hama.apache.org Received: (qmail 43806 invoked by uid 99); 28 Jan 2016 11:33:10 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 28 Jan 2016 11:33:10 +0000 Received: from mail-ig0-f179.google.com (mail-ig0-f179.google.com [209.85.213.179]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 744801A00EA for ; Thu, 28 Jan 2016 11:33:10 +0000 (UTC) Received: by mail-ig0-f179.google.com with SMTP id z14so11298261igp.0 for ; Thu, 28 Jan 2016 03:33:10 -0800 (PST) X-Gm-Message-State: AG10YOQyaKXLb33SJGBkX3555Zw1jwxub16FurqMUOviLoh52WHghy2UuTB2x5UiX3XnmmilbzUvEtARqpPG6Q== MIME-Version: 1.0 X-Received: by 10.50.137.41 with SMTP id qf9mr2646287igb.22.1453980789849; Thu, 28 Jan 2016 03:33:09 -0800 (PST) Received: by 10.64.79.35 with HTTP; Thu, 28 Jan 2016 03:33:09 -0800 (PST) In-Reply-To: References: Date: Thu, 28 Jan 2016 20:33:09 +0900 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: RE: RE: RE: RE: Do Hama support member member variable? From: "Edward J. Yoon" To: "user@hama.apache.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable I filed on https://issues.apache.org/jira/browse/HAMA-982 Once it's fixed, I'll let you know. On Thu, Jan 28, 2016 at 8:11 PM, Edward J. Yoon wro= te: > Wow, you find the bug. Thanks ;) > >> at org.apache.hama.graph.GraphJobRunner$Parser.run(GraphJobRunner.java:= 557) > > When the framework assign vertices to the proper machine at initial > phase, vertex objects are transferred in serialized form. At this step > user defined code won't work correctly. I'll fix soon. > > Anyway, you should able to manage an array of TextPair objects like below= : > > private TextPair[] test =3D new TextPair[1]; > > public void readState(DataInput in) throws IOException { > int size =3D in.readInt(); > test =3D new TextPair[size]; > for(int i =3D 0; i < size; i++) { > test[i].readFields(in); > } > } > > public void writeState(DataOutput out) throws IOException { > out.writeInt(test.length); > for(int i =3D 0; i < test.length; i++) { > test[i].write(out); > } > } > > Thanks. > > > On Thu, Jan 28, 2016 at 6:45 PM, =E6=AD=A5=E9=9D=92=E4=BA=91 wrote: >> Thanks for your reply. You do help me a lot. >> I have tried to use two methods. Some problem are still bothering me. >> When I use the first method of using Hadoop built-in writable classes, I= get a NullPointerException meaning the parents is null=EF=BC=8C even thoug= h I have initialize parents. The code is like this. >> static ArrayWritable parents=3D new ArrayWritable(TextPair.class); >> >> >> public void writeState(DataOutput out) throws IOException { >> out.writeBoolean(match); >> parents.write(out); >> } >> >> >> public void readState(DataInput in) throws IOException { >> match =3D in.readBoolean(); >> parents.readFields(in); >> } >> >> >> >> And the error message is as follow: >> Exception in thread "pool-6-thread-2" java.lang.RuntimeException: java.l= ang.NullPointerException >> at org.apache.hama.graph.GraphJobRunner$Parser.run(GraphJobRunne= r.java:562) >> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolE= xecutor.java:1145) >> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPool= Executor.java:615) >> at java.lang.Thread.run(Thread.java:745) >> Caused by: java.lang.NullPointerException >> at org.apache.hadoop.io.ArrayWritable.write(ArrayWritable.java:1= 03) >> at ProbMatch$ProbMatchVertex.writeState(ProbMatch.java:154) >> at org.apache.hama.graph.Vertex.write(Vertex.java:311) >> at org.apache.hama.util.WritableUtils.unsafeSerialize(WritableUt= ils.java:55) >> at org.apache.hama.graph.MapVerticesInfo.put(MapVerticesInfo.jav= a:64) >> at org.apache.hama.graph.GraphJobRunner.addVertex(GraphJobRunner= .java:577) >> at org.apache.hama.graph.GraphJobRunner.access$300(GraphJobRunne= r.java:64) >> at org.apache.hama.graph.GraphJobRunner$Parser.run(GraphJobRunne= r.java:557) >> >> >> >> When I'm trying use the second method, I don't know how to read and writ= e an object array. I can use out.writeInt() method to write int. But when I= need to write an object. How can I do this? I'm so sorry that I'm not good= at Java. Here is the some code I'm trying to write. >> >> >> static TextPair[] arr =3D new TextPair[1]; >> >> >> public void writeState(DataOutput out) throws IOException { >> out.writeBoolean(match); >> out.writeInt(arr.length); >> for(int i =3D 0; i < arr.length; i++) { >> out.writeInt(arr[i].toString()); // Is this right? >> }} >> >> >> public void readState(DataInput in) throws IOException { >> match =3D in.readBoolean(); >> int length =3D in.readInt(); >> for(int i=3D0;i> arr[i] =3D in.read // How can I read the textpair here? >> } >> } >> >> >> >> I'm very grateful to your help. Thanks again. >> Best Regards, Ping Liu. >> >> >> >> >> ------------------ Original ------------------ >> From: "Edward J. Yoon";; >> Date: Thu, Jan 28, 2016 11:51 AM >> To: "user"; >> >> Subject: RE: RE: RE: RE: Do Hama support member member variable? >> >> >> >> Hi, >> >> You can use Hadoop built-in writable classes or own custom Writable. >> >> static ArrayWritable arr =3D new ArrayWritable(DoubleWritable.class); >> >> public void writeState(DataOutput out) throws IOException { >> arr.write(out); >> } >> >> Or, >> >> static int[] arr2 =3D new int[3]; >> >> public void writeState(DataOutput out) throws IOException { >> out.writeInt(arr2.length); >> for(int i =3D 0; i < arr2.length; i++) { >> out.writeInt(arr2[i]); >> } >> } >> >> public void readState(DataInput in) throws IOException { >> int size =3D in.readInt(); >> for(int i =3D 0; i < size; i++) { >> arr2[i] =3D in.readInt(); >> } >> } >> >> -- >> Best Regards, Edward J. Yoon >> >> >> -----Original Message----- >> From: =E6=AD=A5?=E4=BA=91 [mailto:mailliuping@qq.com] >> Sent: Thursday, January 28, 2016 11:16 AM >> To: user >> Subject: Re: RE: RE: RE: Do Hama support member member variable? >> >> Hi, >> I still don't know how to use member variable. What's the input par= ameter >> DataOutput out in writeState() method? Could you please give me a exampl= e to >> use this method? Thanks a lot. >> By the way, how can I deal with object member variable? For example= , the >> member variable "private List parents" store the parents of the vertex. = How >> can I read and write this variable? >> If you can take time to reply me, I will be very grateful to you. >> Best Regard, Ping Liu. >> >> >> ------------------ Original ------------------ >> From: "Edward J. Yoon";; >> Date: Wed, Jan 27, 2016 03:12 PM >> To: "user"; >> >> Subject: RE: RE: RE: Do Hama support member member variable? >> >> >> >> Just FYI, Hama 0.7.1 release candidate is now available. Please feel fre= e to >> use this, and it'd nice if you can let me know whether it works well wit= h you. >> >> Release tarball: http://people.apache.org/~edwardyoon/dist/0.7.1-RC1 >> >> -- >> Best Regards, Edward J. Yoon >> >> >> -----Original Message----- >> From: =E6=AD=A5?=E4=BA=91 [mailto:mailliuping@qq.com] >> Sent: Wednesday, January 27, 2016 11:01 AM >> To: user >> Subject: Re: RE: RE: Do Hama support member member variable? >> >> I know. If I use the static variable like "private static boolean match"= , I >> could get the right value of match too. Could I use static variable? >> Thanks a lot. >> Best regards. Ping Liu. >> >> >> >> >> ------------------ Original ------------------ >> From: "Edward J. Yoon";; >> Date: Wed, Jan 27, 2016 09:39 AM >> To: "user"; >> >> Subject: RE: RE: Do Hama support member member variable? >> >> >> >> Basically Vertex object is writable, and we store the vertex objects in >> serialized form. There are two purposes: 1) to reduce memory usage 2) to= write >> on file system (checkpoint and recovery). >> >> So, you should use readState() and writeState() methods to save object m= ember >> variables. >> >> Thanks! >> >> -- >> Best Regards, Edward J. Yoon >> >> >> -----Original Message----- >> From: =E6=AD=A5?=E4=BA=91 [mailto:mailliuping@qq.com] >> Sent: Wednesday, January 27, 2016 10:27 AM >> To: user >> Subject: Re: RE: Do Hama support member member variable? >> >> Hi, >> Thank you very much. You helped me a lot. But I still don't know how to = use >> readState() and writeState() methods. What is the input parameter DataIn= put in >> and DataOutput out? And as far as I know, I can assign the value of mem= ber >> variable directly in hama-0.6.3, such as match =3D true. Why I can't do = this in >> Hama-0.7.0? >> Waiting for your reply. Thanks. >> Best Regards, Ping Liu. >> >> >> ------------------ Original ------------------ >> From: "Edward J. Yoon";; >> Date: Wed, Jan 27, 2016 07:15 AM >> To: "user"; >> >> Subject: RE: Do Hama support member member variable? >> >> >> >> Hi, >> >> You should use readState() and writeState() methods like below: >> >> public static class ProbMatchVertex extends Vertex> TriTextPair> { >> private boolean match =3D false; >> >> public void readState(DataInput in) throws IOException { >> match =3D in.readBoolean(); >> } >> >> public void writeState(DataOutput out) throws IOException { >> out.writeBoolean(match); >> } >> >> .. >> >> } >> >> -- >> Best Regards, Edward J. Yoon >> >> -----Original Message----- >> From: =EA=BC=8D=ED=96=89=E6=9A=BE [mailto:mailliuping@qq.com] >> Sent: Tuesday, January 26, 2016 10:16 PM >> To: user >> Subject: Do Hama support member member variable? >> >> Hello, >> I'm trying to run a graph job. But i have got some problems. >> I want to use member variable in vertex class, the code is as foll= ow. >> I have changed the value of the member variable in one superstep. But wh= en I >> use this member variable in next superstep, the value of this member >> variable is still the defalult value. For example, "match" is the member >> variable. I have changed the value of "match" to be ture in superstep 0,= but >> when I print "match" in superstep 1, the result was "match: false". >> Could anyone tell me why the value of member variable is changed? >> Thanks very much. >> Best wishes. >> >> >> public static class ProbMatchVertex extends Vertex> TriTextPair> { >> private boolean match =3D false; >> >> >> @Override >> public void compute(Iterable messages) thro= ws >> IOException { >> if (getSuperstepCount() =3D=3D 0) { >> match =3D true; >> sendMessageToNeighbors(new >> TriTextPair(getVertexID(), >> getVertexLabel(), new >> Text(""))); >> } else if(getSuperstepCount() =3D=3D 1){ >> System.out.println("match:" + match); >> parents =3D new ArrayList()= ; >> for(TriTextPair msg : messages){ >> parents.add(msg); >> sendMessage(msg.getFirst(), new >> TriTextPair(getVertexID(), getVertexLabel(), new Text(""))); >> } >> } >> } >> >> >> >> >> Ping Liu > > > > -- > Best Regards, Edward J. Yoon --=20 Best Regards, Edward J. Yoon