Return-Path: X-Original-To: apmail-giraph-user-archive@www.apache.org Delivered-To: apmail-giraph-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id ED881ECDA for ; Thu, 21 Feb 2013 23:25:22 +0000 (UTC) Received: (qmail 65651 invoked by uid 500); 21 Feb 2013 23:25:22 -0000 Delivered-To: apmail-giraph-user-archive@giraph.apache.org Received: (qmail 65614 invoked by uid 500); 21 Feb 2013 23:25:22 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 65605 invoked by uid 99); 21 Feb 2013 23:25:22 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 21 Feb 2013 23:25:22 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of prvs=5764bc1ab4=majakabiljo@fb.com designates 67.231.145.42 as permitted sender) Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com) (67.231.145.42) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 21 Feb 2013 23:25:15 +0000 Received: from pps.filterd (m0004346 [127.0.0.1]) by mx0a-00082601.pphosted.com (8.14.5/8.14.5) with SMTP id r1LNMrXW003138 for ; Thu, 21 Feb 2013 15:24:53 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=fb.com; h=from : to : subject : date : message-id : in-reply-to : content-type : mime-version; s=facebook; bh=veXcVNWCGzEkcZrMweJjbUe6TGAIKdPbi/OByiRJzek=; b=U1kkhD4UMu6WG+PyI6GDWI4IovV0VeSI+kiTXWBMJCq/QnyT8xNZzX0tZMS6NlZogYiR KkXybddo0orKi5+IXURnjA70OXieJSgDy+dicGulZMrLr70B980WQHZtpWBiwm4MHOCt 1YSxPCBvx0KjCBpu6qAHUNkM312aDhheAMU= Received: from mail.thefacebook.com (prn1-cmdf-dc01-fw1-nat.corp.tfbnw.net [173.252.71.129] (may be forged)) by mx0a-00082601.pphosted.com with ESMTP id 1anjfqhy5e-3 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=OK) for ; Thu, 21 Feb 2013 15:24:53 -0800 Received: from PRN-MBX02-2.TheFacebook.com ([169.254.5.107]) by PRN-CHUB01.TheFacebook.com ([fe80::d5cc:849:f520:db6b%12]) with mapi id 14.02.0328.011; Thu, 21 Feb 2013 15:24:09 -0800 From: Maja Kabiljo To: "user@giraph.apache.org" Subject: Re: InputFormat for the example SimpleMasterComputeVertex Thread-Topic: InputFormat for the example SimpleMasterComputeVertex Thread-Index: AQHOCdn7Jbe13kZZoUS1HpoRJa0vy5h6djyA//+jg4CACxo/gP//zWSA Date: Thu, 21 Feb 2013 23:24:08 +0000 Message-ID: <1F592C080E9ACB4CB1C9EA1865BF3EFA056028B7@PRN-MBX02-2.TheFacebook.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.16.4] Content-Type: multipart/alternative; boundary="_000_1F592C080E9ACB4CB1C9EA1865BF3EFA056028B7PRNMBX022TheFac_" MIME-Version: 1.0 X-Proofpoint-Spam-Reason: safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.9.8327,1.0.431,0.0.0000 definitions=2013-02-21_09:2013-02-21,2013-02-21,1970-01-01 signatures=0 X-Virus-Checked: Checked by ClamAV on apache.org --_000_1F592C080E9ACB4CB1C9EA1865BF3EFA056028B7PRNMBX022TheFac_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Eli, that's an interesting idea, we could have some class which user extend= s and which is there only for aggregator registration. Sometimes we want to= register some aggregators later on during the computation, so we need to k= eep allowing registration from masterCompute too. But I think for users the biggest problem is to realize that they have to e= xtend and set MasterCompute/this new class in order to use aggregators. Cur= rently, if user tries to aggregate a value to unregistered aggregator he wi= ll get an exception, but if he tries to get the value of unregistered aggre= gator he will just get null. So maybe adding a warning message in that case= , together with a wiki page, might be enough? What do you think? From: Eli Reisman > Reply-To: "user@giraph.apache.org" > Date: Thursday, February 21, 2013 10:25 AM To: "user@giraph.apache.org" > Subject: Re: InputFormat for the example SimpleMasterComputeVertex Thanks for the explanation, that makes sense. I would love to see a wiki pa= ge at some point, you have so much knowledge of this piece of Giraph from a= ll your dev work on it and have also the additional bonus of experience run= ning big cluster jobs using these features so you have a lot of insight to = share. Would there be any point to a future JIRA to break out the aggregator regis= tration from the master compute stuff, at least from the user's view? Or is= it not that confusing once you've used them a bit? On Thu, Feb 14, 2013 at 4:52 PM, Maja Kabiljo > wrote: Progressable exception can be caused by many different reasons (it's totall= y unrelated to aggregators), and when looking at which exception it's cause= d by users should get better sense about what's going on. What you are suggesting about providing default master compute is not doabl= e, since the part which needs to be done there is aggregator registration. = We can't know what kind of aggregators (names and types) an application nee= ds. I remember I was talking about writing a short tutorial for aggregators lon= g time ago, sorry for not doing that, will try to get to it soon. From: Eli Reisman > Reply-To: "user@giraph.apache.org" > Date: Thursday, February 14, 2013 2:23 PM To: "user@giraph.apache.org" > Subject: Re: InputFormat for the example SimpleMasterComputeVertex Other folks on the list are also having this problem with the progressable = utile exception & job failures. I don't know much about master compute usag= e but if it is needed to make the aggregators work, maybe we should have a = default dummy class that just handles aggregators if no other master comput= e is specified? Or a wiki page? The progressable error message does not lea= d us to this conclusion directly. On Wed, Feb 13, 2013 at 3:04 AM, Maria Stylianou > wrote: Hey, I am trying to run the example SimpleMasterComputeVertex, but no matter whi= ch Input Format and graph I give, it doesn't work. Each worker gives the er= ror: Caused by: java.lang.NullPointerException at org.apache.giraph.examples.SimpleMasterComputeVertex.compute(Sim= pleMasterComputeVertex.java:42) This line 42 is the first line of the compute() public void compute(Iterable messages){ So I guess, the initialization is not done correctly, because the input fil= e does not have the correct format. Any help would be appreciated, Thanks! Maria -- Maria Stylianou Intern at Telefonica, Barcelona, Spain Master Student of European Master in Distributed Computing Universitat Polit=E8cnica de Catalunya - BarcelonaTech, Barcelona, Spain KTH Royal Institute of Technology, Stockholm, Sweden --_000_1F592C080E9ACB4CB1C9EA1865BF3EFA056028B7PRNMBX022TheFac_ Content-Type: text/html; charset="iso-8859-1" Content-ID: <072720DDF04629429F60B07A74607D16@fb.com> Content-Transfer-Encoding: quoted-printable
Eli, that's an interesting idea, we could have some class which user e= xtends and which is there only for aggregator registration. Sometimes we wa= nt to register some aggregators later on during the computation, so we need= to keep allowing registration from masterCompute too. 

But I think for users the biggest problem is to realize that they have= to extend and set MasterCompute/this new class in order to use aggregators= . Currently, if user tries to aggregate a value to unregistered aggregator = he will get an exception, but if he tries to get the value of unregistered aggregator he will just get null= . So maybe adding a warning message in that case, together with a wiki page= , might be enough? What do you think?

From: Eli Reisman <apache.mailbox@gmail.com>
Reply-To: "user@giraph.apache.org" <user@giraph.apache.org>
Date: Thursday, February 21, 2013 1= 0:25 AM
To: "user@giraph.apache.org" <user@giraph.apache.org>
Subject: Re: InputFormat for the ex= ample SimpleMasterComputeVertex

Thanks for the explanation, that makes sense. I would love to see a wi= ki page at some point, you have so much knowledge of this piece of Giraph f= rom all your dev work on it and have also the additional bonus of experienc= e running big cluster jobs using these features so you have a lot of insight to share.

Would there be any point to a future JIRA to break out the aggregator = registration from the master compute stuff, at least from the user's view? = Or is it not that confusing once you've used them a bit?


On Thu, Feb 14, 2013 at 4:52 PM, Maja Kabiljo <majakabiljo@fb.= com> wrote:
Progressable exception can be caused by many different reasons (it's t= otally unrelated to aggregators), and when looking at which exception it's = caused by users should get better sense about what's going on. 
What you are suggesting about providing default master compute is not = doable, since the part which needs to be done there is aggregator registrat= ion. We can't know what kind of aggregators (names and types) an applicatio= n needs.
I remember I was talking about writing a short tutorial for aggregator= s long time ago, sorry for not doing that, will try to get to it soon.

From: Eli Reisman <apache.mailbox@gmail.com>
Reply-To: "user@giraph.apache.org" &= lt;user@giraph.= apache.org>
Date: Thursday, February 14, 2013 2= :23 PM

To: "user@giraph.apache.org" <user@giraph.apache= .org>
Subject: Re: InputFormat for the ex= ample SimpleMasterComputeVertex

Other folks on the list are also having this problem with the progress= able utile exception & job failures. I don't know much about maste= r compute usage but if it is needed to make the aggregators work, maybe we = should have a default dummy class that just handles aggregators if no other master compute is specified? Or a wiki pag= e? The progressable error message does not lead us to this conclusion direc= tly.

On Wed, Feb 13, 2013 at 3:04 AM, Maria Stylianou= <marsty5@gmail.co= m> wrote:
Hey,

I am trying to run the example SimpleMasterComputeVertex, but no matte= r which Input Format and graph I give, it doesn't work. Each worker gives t= he error: 
Ca=
used by: java.lang.NullPointerException
	at org.apache.giraph.examples.SimpleMasterComputeVertex.compute(Sim=
pleMasterComputeVertex.java:42)
This line 42 is the first line of the compute() 
pu= blic void compute(Iterable<DoubleWritable> messages){&n= bsp;

So I guess, the initialization is not done correctly, because the inpu= t file does not have the correct format.

Any help would be appreciated,
Thanks!
Maria
--
Maria Stylianou
Intern at Telefonica, Barcelona, Spain
Master Student of European Master in Distributed Computing
Universitat Polit=E8cnica de Catalunya - BarcelonaTech, Barcelona, Spa= in
KTH Royal Institute of Technology, Stockholm, Sweden



 --_000_1F592C080E9ACB4CB1C9EA1865BF3EFA056028B7PRNMBX022TheFac_--