giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eli Reisman <>
Subject Re: InputFormat for the example SimpleMasterComputeVertex
Date Sat, 02 Mar 2013 21:57:07 GMT
Thanks for doing this Maja!

On Thu, Feb 21, 2013 at 5:12 PM, Maja Kabiljo <> wrote:

>  I wrote up a basic info about aggregators:
> Please
> take a look, and let me know if something needs to be changed / improved.
>   From: Eli Reisman <>
> Reply-To: "" <>
> Date: Thursday, February 21, 2013 4:21 PM
> To: "" <>
> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>  That sounds great to me, maybe just a mention in the wiki that the two
> functionalities are tied together will help the idea click for people.
> Either way this will be a big help I think.
> On Thu, Feb 21, 2013 at 3:24 PM, Maja Kabiljo <> wrote:
>>  Eli, that's an interesting idea, we could have some class which user
>> extends and which is there only for aggregator registration. Sometimes we
>> want to register some aggregators later on during the computation, so we
>> need to keep allowing registration from masterCompute too.
>>  But I think for users the biggest problem is to realize that they have
>> to extend and set MasterCompute/this new class in order to use aggregators.
>> Currently, if user tries to aggregate a value to unregistered aggregator he
>> will get an exception, but if he tries to get the value of unregistered
>> aggregator he will just get null. So maybe adding a warning message in that
>> case, together with a wiki page, might be enough? What do you think?
>>   From: Eli Reisman <>
>> Reply-To: "" <>
>>  Date: Thursday, February 21, 2013 10:25 AM
>> To: "" <>
>> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>>  Thanks for the explanation, that makes sense. I would love to see a
>> wiki page at some point, you have so much knowledge of this piece of Giraph
>> from all your dev work on it and have also the additional bonus of
>> experience running big cluster jobs using these features so you have a lot
>> of insight to share.
>>  Would there be any point to a future JIRA to break out the aggregator
>> registration from the master compute stuff, at least from the user's view?
>> Or is it not that confusing once you've used them a bit?
>> On Thu, Feb 14, 2013 at 4:52 PM, Maja Kabiljo <> wrote:
>>>  Progressable exception can be caused by many different reasons (it's
>>> totally unrelated to aggregators), and when looking at which exception it's
>>> caused by users should get better sense about what's going on.
>>> What you are suggesting about providing default master compute is not
>>> doable, since the part which needs to be done there is aggregator
>>> registration. We can't know what kind of aggregators (names and types) an
>>> application needs.
>>> I remember I was talking about writing a short tutorial for aggregators
>>> long time ago, sorry for not doing that, will try to get to it soon.
>>>   From: Eli Reisman <>
>>> Reply-To: "" <>
>>> Date: Thursday, February 14, 2013 2:23 PM
>>> To: "" <>
>>> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>>>   Other folks on the list are also having this problem with the
>>> progressable utile exception & job failures. I don't know much about master
>>> compute usage but if it is needed to make the aggregators work, maybe we
>>> should have a default dummy class that just handles aggregators if no other
>>> master compute is specified? Or a wiki page? The progressable error message
>>> does not lead us to this conclusion directly.
>>> On Wed, Feb 13, 2013 at 3:04 AM, Maria Stylianou <>wrote:
>>>> Hey,
>>>>  I am trying to run the example SimpleMasterComputeVertex, but no
>>>> matter which Input Format and graph I give, it doesn't work. Each worker
>>>> gives the error:
>>>> Caused by: java.lang.*NullPointerException*
>>>> 	at org.apache.giraph.examples.*SimpleMasterComputeVertex.compute*(*42*)
>>>> This line 42 is the first line of the compute()
>>>> public void compute(Iterable<DoubleWritable> messages){
>>>>  So I guess, the initialization is not done correctly, because the
>>>> input file does not have the correct format.
>>>>  Any help would be appreciated,
>>>> Thanks!
>>>>  Maria
>>>> --
>>>> Maria Stylianou
>>>> Intern at Telefonica, Barcelona, Spain
>>>> Master Student of European Master in Distributed Computing<>
>>>> Universitat Polit├Ęcnica de Catalunya - BarcelonaTech, Barcelona, Spain
>>>>  KTH Royal Institute of Technology, Stockholm, Sweden

View raw message