giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eli Reisman <apache.mail...@gmail.com>
Subject Re: InputFormat for the example SimpleMasterComputeVertex
Date Sat, 02 Mar 2013 21:57:07 GMT
Thanks for doing this Maja!

On Thu, Feb 21, 2013 at 5:12 PM, Maja Kabiljo <majakabiljo@fb.com> wrote:

>  I wrote up a basic info about aggregators:
> https://cwiki.apache.org/confluence/display/GIRAPH/Aggregators. Please
> take a look, and let me know if something needs to be changed / improved.
>
>   From: Eli Reisman <apache.mailbox@gmail.com>
> Reply-To: "user@giraph.apache.org" <user@giraph.apache.org>
> Date: Thursday, February 21, 2013 4:21 PM
>
> To: "user@giraph.apache.org" <user@giraph.apache.org>
> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>
>  That sounds great to me, maybe just a mention in the wiki that the two
> functionalities are tied together will help the idea click for people.
> Either way this will be a big help I think.
>
>
> On Thu, Feb 21, 2013 at 3:24 PM, Maja Kabiljo <majakabiljo@fb.com> wrote:
>
>>  Eli, that's an interesting idea, we could have some class which user
>> extends and which is there only for aggregator registration. Sometimes we
>> want to register some aggregators later on during the computation, so we
>> need to keep allowing registration from masterCompute too.
>>
>>  But I think for users the biggest problem is to realize that they have
>> to extend and set MasterCompute/this new class in order to use aggregators.
>> Currently, if user tries to aggregate a value to unregistered aggregator he
>> will get an exception, but if he tries to get the value of unregistered
>> aggregator he will just get null. So maybe adding a warning message in that
>> case, together with a wiki page, might be enough? What do you think?
>>
>>   From: Eli Reisman <apache.mailbox@gmail.com>
>> Reply-To: "user@giraph.apache.org" <user@giraph.apache.org>
>>  Date: Thursday, February 21, 2013 10:25 AM
>>
>> To: "user@giraph.apache.org" <user@giraph.apache.org>
>> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>>
>>  Thanks for the explanation, that makes sense. I would love to see a
>> wiki page at some point, you have so much knowledge of this piece of Giraph
>> from all your dev work on it and have also the additional bonus of
>> experience running big cluster jobs using these features so you have a lot
>> of insight to share.
>>
>>  Would there be any point to a future JIRA to break out the aggregator
>> registration from the master compute stuff, at least from the user's view?
>> Or is it not that confusing once you've used them a bit?
>>
>>
>> On Thu, Feb 14, 2013 at 4:52 PM, Maja Kabiljo <majakabiljo@fb.com> wrote:
>>
>>>  Progressable exception can be caused by many different reasons (it's
>>> totally unrelated to aggregators), and when looking at which exception it's
>>> caused by users should get better sense about what's going on.
>>> What you are suggesting about providing default master compute is not
>>> doable, since the part which needs to be done there is aggregator
>>> registration. We can't know what kind of aggregators (names and types) an
>>> application needs.
>>> I remember I was talking about writing a short tutorial for aggregators
>>> long time ago, sorry for not doing that, will try to get to it soon.
>>>
>>>   From: Eli Reisman <apache.mailbox@gmail.com>
>>> Reply-To: "user@giraph.apache.org" <user@giraph.apache.org>
>>> Date: Thursday, February 14, 2013 2:23 PM
>>>
>>> To: "user@giraph.apache.org" <user@giraph.apache.org>
>>> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>>>
>>>   Other folks on the list are also having this problem with the
>>> progressable utile exception & job failures. I don't know much about master
>>> compute usage but if it is needed to make the aggregators work, maybe we
>>> should have a default dummy class that just handles aggregators if no other
>>> master compute is specified? Or a wiki page? The progressable error message
>>> does not lead us to this conclusion directly.
>>>
>>> On Wed, Feb 13, 2013 at 3:04 AM, Maria Stylianou <marsty5@gmail.com>wrote:
>>>
>>>> Hey,
>>>>
>>>>  I am trying to run the example SimpleMasterComputeVertex, but no
>>>> matter which Input Format and graph I give, it doesn't work. Each worker
>>>> gives the error:
>>>>
>>>> Caused by: java.lang.*NullPointerException*
>>>> 	at org.apache.giraph.examples.*SimpleMasterComputeVertex.compute*(SimpleMasterComputeVertex.java:*42*)
>>>>
>>>> This line 42 is the first line of the compute()
>>>> public void compute(Iterable<DoubleWritable> messages){
>>>>
>>>>  So I guess, the initialization is not done correctly, because the
>>>> input file does not have the correct format.
>>>>
>>>>  Any help would be appreciated,
>>>> Thanks!
>>>>  Maria
>>>> --
>>>> Maria Stylianou
>>>> Intern at Telefonica, Barcelona, Spain
>>>> Master Student of European Master in Distributed Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
>>>> Universitat Polit├Ęcnica de Catalunya - BarcelonaTech, Barcelona, Spain
>>>>  KTH Royal Institute of Technology, Stockholm, Sweden
>>>>
>>>>
>>>
>>
>

Mime
View raw message