Return-Path: X-Original-To: apmail-giraph-user-archive@www.apache.org Delivered-To: apmail-giraph-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 710D2E859 for ; Fri, 22 Feb 2013 01:13:26 +0000 (UTC) Received: (qmail 2584 invoked by uid 500); 22 Feb 2013 01:13:26 -0000 Delivered-To: apmail-giraph-user-archive@giraph.apache.org Received: (qmail 2530 invoked by uid 500); 22 Feb 2013 01:13:26 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 2522 invoked by uid 99); 22 Feb 2013 01:13:26 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 22 Feb 2013 01:13:26 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of prvs=576520bc08=majakabiljo@fb.com designates 67.231.145.42 as permitted sender) Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com) (67.231.145.42) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 22 Feb 2013 01:13:18 +0000 Received: from pps.filterd (m0004346 [127.0.0.1]) by mx0a-00082601.pphosted.com (8.14.5/8.14.5) with SMTP id r1M1BukA027445 for ; Thu, 21 Feb 2013 17:12:56 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fb.com; h=from : to : subject : date : message-id : in-reply-to : content-type : mime-version; s=facebook; bh=mHJYMFU0jdgzKnapF7nKvFWc4iQFjd+W4l3Ony/0+3w=; b=ePqfZW375ZbhDwvvky7mkAFPfbV1f+u4PcsuJ15/7wmqUUBuj/YqlDr5WplzUFIIYo9P XOH6UZyQbyq7JNrf8OJQZBBIwQGVQLIJbfmy2Rj2AyrCYZNjXzhtl1z3m9ySFGTzaPEM 6m23A/RlRvT/6vVmAII+UNSuQfLn4tsIplw= Received: from mail.thefacebook.com (prn1-cmdf-dc01-fw1-nat.corp.tfbnw.net [173.252.71.129] (may be forged)) by mx0a-00082601.pphosted.com with ESMTP id 1anjfqjb5h-1 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=OK) for ; Thu, 21 Feb 2013 17:12:56 -0800 Received: from PRN-MBX02-2.TheFacebook.com ([169.254.5.107]) by PRN-CHUB02.TheFacebook.com ([fe80::5de8:34:5a87:6990%12]) with mapi id 14.02.0328.011; Thu, 21 Feb 2013 17:12:55 -0800 From: Maja Kabiljo To: "user@giraph.apache.org" Subject: Re: InputFormat for the example SimpleMasterComputeVertex Thread-Topic: InputFormat for the example SimpleMasterComputeVertex Thread-Index: AQHOCdn7Jbe13kZZoUS1HpoRJa0vy5h6djyA//+jg4CACxo/gP//zWSAgACWOgD//4gpAA== Date: Fri, 22 Feb 2013 01:12:54 +0000 Message-ID: <1F592C080E9ACB4CB1C9EA1865BF3EFA05602E85@PRN-MBX02-2.TheFacebook.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.16.4] Content-Type: multipart/alternative; boundary="_000_1F592C080E9ACB4CB1C9EA1865BF3EFA05602E85PRNMBX022TheFac_" MIME-Version: 1.0 X-Proofpoint-Spam-Reason: safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.9.8327,1.0.431,0.0.0000 definitions=2013-02-21_09:2013-02-21,2013-02-21,1970-01-01 signatures=0 X-Virus-Checked: Checked by ClamAV on apache.org --_000_1F592C080E9ACB4CB1C9EA1865BF3EFA05602E85PRNMBX022TheFac_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable I wrote up a basic info about aggregators: https://cwiki.apache.org/conflue= nce/display/GIRAPH/Aggregators. Please take a look, and let me know if some= thing needs to be changed / improved. From: Eli Reisman > Reply-To: "user@giraph.apache.org" > Date: Thursday, February 21, 2013 4:21 PM To: "user@giraph.apache.org" > Subject: Re: InputFormat for the example SimpleMasterComputeVertex That sounds great to me, maybe just a mention in the wiki that the two func= tionalities are tied together will help the idea click for people. Either w= ay this will be a big help I think. On Thu, Feb 21, 2013 at 3:24 PM, Maja Kabiljo > wrote: Eli, that's an interesting idea, we could have some class which user extend= s and which is there only for aggregator registration. Sometimes we want to= register some aggregators later on during the computation, so we need to k= eep allowing registration from masterCompute too. But I think for users the biggest problem is to realize that they have to e= xtend and set MasterCompute/this new class in order to use aggregators. Cur= rently, if user tries to aggregate a value to unregistered aggregator he wi= ll get an exception, but if he tries to get the value of unregistered aggre= gator he will just get null. So maybe adding a warning message in that case= , together with a wiki page, might be enough? What do you think? From: Eli Reisman > Reply-To: "user@giraph.apache.org" > Date: Thursday, February 21, 2013 10:25 AM To: "user@giraph.apache.org" > Subject: Re: InputFormat for the example SimpleMasterComputeVertex Thanks for the explanation, that makes sense. I would love to see a wiki pa= ge at some point, you have so much knowledge of this piece of Giraph from a= ll your dev work on it and have also the additional bonus of experience run= ning big cluster jobs using these features so you have a lot of insight to = share. Would there be any point to a future JIRA to break out the aggregator regis= tration from the master compute stuff, at least from the user's view? Or is= it not that confusing once you've used them a bit? On Thu, Feb 14, 2013 at 4:52 PM, Maja Kabiljo > wrote: Progressable exception can be caused by many different reasons (it's totall= y unrelated to aggregators), and when looking at which exception it's cause= d by users should get better sense about what's going on. What you are suggesting about providing default master compute is not doabl= e, since the part which needs to be done there is aggregator registration. = We can't know what kind of aggregators (names and types) an application nee= ds. I remember I was talking about writing a short tutorial for aggregators lon= g time ago, sorry for not doing that, will try to get to it soon. From: Eli Reisman > Reply-To: "user@giraph.apache.org" > Date: Thursday, February 14, 2013 2:23 PM To: "user@giraph.apache.org" > Subject: Re: InputFormat for the example SimpleMasterComputeVertex Other folks on the list are also having this problem with the progressable = utile exception & job failures. I don't know much about master compute usag= e but if it is needed to make the aggregators work, maybe we should have a = default dummy class that just handles aggregators if no other master comput= e is specified? Or a wiki page? The progressable error message does not lea= d us to this conclusion directly. On Wed, Feb 13, 2013 at 3:04 AM, Maria Stylianou > wrote: Hey, I am trying to run the example SimpleMasterComputeVertex, but no matter whi= ch Input Format and graph I give, it doesn't work. Each worker gives the er= ror: Caused by: java.lang.NullPointerException at org.apache.giraph.examples.SimpleMasterComputeVertex.compute(Sim= pleMasterComputeVertex.java:42) This line 42 is the first line of the compute() public void compute(Iterable messages){ So I guess, the initialization is not done correctly, because the input fil= e does not have the correct format. Any help would be appreciated, Thanks! Maria -- Maria Stylianou Intern at Telefonica, Barcelona, Spain Master Student of European Master in Distributed Computing Universitat Polit=E8cnica de Catalunya - BarcelonaTech, Barcelona, Spain KTH Royal Institute of Technology, Stockholm, Sweden --_000_1F592C080E9ACB4CB1C9EA1865BF3EFA05602E85PRNMBX022TheFac_ Content-Type: text/html; charset="iso-8859-1" Content-ID: <4F327F0A3A74334EA66662B72C097A15@fb.com> Content-Transfer-Encoding: quoted-printable
I wrote up a basic info about aggregators: https://cwiki.apache.o= rg/confluence/display/GIRAPH/Aggregators. Please take a look, and let m= e know if something needs to be changed / improved.

From: Eli Reisman <apache.mailbox@gmail.com>
Reply-To: "user@giraph.apache.org" <user@giraph.apache.org>
Date: Thursday, February 21, 2013 4= :21 PM
To: "user@giraph.apache.org" <user@giraph.apache.org>
Subject: Re: InputFormat for the ex= ample SimpleMasterComputeVertex

That sounds great to me, maybe just a mention in the wiki that the two= functionalities are tied together will help the idea click for people. Eit= her way this will be a big help I think.


On Thu, Feb 21, 2013 at 3:24 PM, Maja Kabiljo <majakabiljo@fb.= com> wrote:
Eli, that's an interesting idea, we could have some class which user e= xtends and which is there only for aggregator registration. Sometimes we wa= nt to register some aggregators later on during the computation, so we need= to keep allowing registration from masterCompute too. 

But I think for users the biggest problem is to realize that they have= to extend and set MasterCompute/this new class in order to use aggregators= . Currently, if user tries to aggregate a value to unregistered aggregator = he will get an exception, but if he tries to get the value of unregistered aggregator he will just get null= . So maybe adding a warning message in that case, together with a wiki page= , might be enough? What do you think?

From: Eli Reisman= <apache.m= ailbox@gmail.com>
Reply-To: "user@giraph.apache.org" &= lt;user@giraph.= apache.org>
Date: Thursday, February 21, 2013 1= 0:25 AM

To: "user@giraph.apache.org" <user@giraph.apache= .org>
Subject: Re: InputFormat for the ex= ample SimpleMasterComputeVertex

Thanks for the explanation, that makes sense. I would love to see a wi= ki page at some point, you have so much knowledge of this piece of Giraph f= rom all your dev work on it and have also the additional bonus of experienc= e running big cluster jobs using these features so you have a lot of insight to share.

Would there be any point to a future JIRA to break out the aggregator = registration from the master compute stuff, at least from the user's view? = Or is it not that confusing once you've used them a bit?


On Thu, Feb 14, 2013 at 4:52 PM, Maja Kabiljo <majakabiljo@fb.= com> wrote:
Progressable exception can be caused by many different reasons (it's t= otally unrelated to aggregators), and when looking at which exception it's = caused by users should get better sense about what's going on. 
What you are suggesting about providing default master compute is not = doable, since the part which needs to be done there is aggregator registrat= ion. We can't know what kind of aggregators (names and types) an applicatio= n needs.
I remember I was talking about writing a short tutorial for aggregator= s long time ago, sorry for not doing that, will try to get to it soon.

From: Eli Reisman <apache.mailbox@gmail.com>
Reply-To: "user@giraph.apache.org" &= lt;user@giraph.= apache.org>
Date: Thursday, February 14, 2013 2= :23 PM

To: "user@giraph.apache.org" <user@giraph.apache= .org>
Subject: Re: InputFormat for the ex= ample SimpleMasterComputeVertex

Other folks on the list are also having this problem with the progress= able utile exception & job failures. I don't know much about maste= r compute usage but if it is needed to make the aggregators work, maybe we = should have a default dummy class that just handles aggregators if no other master compute is specified? Or a wiki pag= e? The progressable error message does not lead us to this conclusion direc= tly.

On Wed, Feb 13, 2013 at 3:04 AM, Maria Stylianou= <marsty5@gmail.co= m> wrote:
Hey,

I am trying to run the example SimpleMasterComputeVertex, but no matte= r which Input Format and graph I give, it doesn't work. Each worker gives t= he error: 
Ca=
used by: java.lang.NullPointerException
	at org.apache.giraph.examples.SimpleMasterComputeVertex.compute(Sim=
pleMasterComputeVertex.java:42)
This line 42 is the first line of the compute() 
pu= blic void compute(Iterable<DoubleWritable> messages){&n= bsp;

So I guess, the initialization is not done correctly, because the inpu= t file does not have the correct format.

Any help would be appreciated,
Thanks!
Maria
--
Maria Stylianou
Intern at Telefonica, Barcelona, Spain
Master Student of European Master in Distributed Computing
Universitat Polit=E8cnica de Catalunya - BarcelonaTech, Barcelona, Spa= in
KTH Royal Institute of Technology, Stockholm, Sweden




 --_000_1F592C080E9ACB4CB1C9EA1865BF3EFA05602E85PRNMBX022TheFac_--