giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eli Reisman <>
Subject Re: InputFormat for the example SimpleMasterComputeVertex
Date Sat, 02 Mar 2013 22:01:03 GMT
Its fantastic thanks again! We need more wiki pages like that!!! This will
save so many emails on the list too! I need to (and will) update the graph
representations page to say we support multigraph, edge and vertex IO, etc.
soon. A lot of our wiki info is old enough to be confusing people now.

On Sat, Mar 2, 2013 at 1:57 PM, Eli Reisman <>wrote:

> Thanks for doing this Maja!
> On Thu, Feb 21, 2013 at 5:12 PM, Maja Kabiljo <> wrote:
>>  I wrote up a basic info about aggregators:
>> Please
>> take a look, and let me know if something needs to be changed / improved.
>>   From: Eli Reisman <>
>> Reply-To: "" <>
>> Date: Thursday, February 21, 2013 4:21 PM
>> To: "" <>
>> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>>  That sounds great to me, maybe just a mention in the wiki that the two
>> functionalities are tied together will help the idea click for people.
>> Either way this will be a big help I think.
>> On Thu, Feb 21, 2013 at 3:24 PM, Maja Kabiljo <> wrote:
>>>  Eli, that's an interesting idea, we could have some class which user
>>> extends and which is there only for aggregator registration. Sometimes we
>>> want to register some aggregators later on during the computation, so we
>>> need to keep allowing registration from masterCompute too.
>>>  But I think for users the biggest problem is to realize that they have
>>> to extend and set MasterCompute/this new class in order to use aggregators.
>>> Currently, if user tries to aggregate a value to unregistered aggregator he
>>> will get an exception, but if he tries to get the value of unregistered
>>> aggregator he will just get null. So maybe adding a warning message in that
>>> case, together with a wiki page, might be enough? What do you think?
>>>   From: Eli Reisman <>
>>> Reply-To: "" <>
>>>  Date: Thursday, February 21, 2013 10:25 AM
>>> To: "" <>
>>> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>>>  Thanks for the explanation, that makes sense. I would love to see a
>>> wiki page at some point, you have so much knowledge of this piece of Giraph
>>> from all your dev work on it and have also the additional bonus of
>>> experience running big cluster jobs using these features so you have a lot
>>> of insight to share.
>>>  Would there be any point to a future JIRA to break out the aggregator
>>> registration from the master compute stuff, at least from the user's view?
>>> Or is it not that confusing once you've used them a bit?
>>> On Thu, Feb 14, 2013 at 4:52 PM, Maja Kabiljo <>wrote:
>>>>  Progressable exception can be caused by many different reasons (it's
>>>> totally unrelated to aggregators), and when looking at which exception it's
>>>> caused by users should get better sense about what's going on.
>>>> What you are suggesting about providing default master compute is not
>>>> doable, since the part which needs to be done there is aggregator
>>>> registration. We can't know what kind of aggregators (names and types) an
>>>> application needs.
>>>> I remember I was talking about writing a short tutorial for aggregators
>>>> long time ago, sorry for not doing that, will try to get to it soon.
>>>>   From: Eli Reisman <>
>>>> Reply-To: "" <>
>>>> Date: Thursday, February 14, 2013 2:23 PM
>>>> To: "" <>
>>>> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>>>>   Other folks on the list are also having this problem with the
>>>> progressable utile exception & job failures. I don't know much about
>>>> compute usage but if it is needed to make the aggregators work, maybe we
>>>> should have a default dummy class that just handles aggregators if no other
>>>> master compute is specified? Or a wiki page? The progressable error message
>>>> does not lead us to this conclusion directly.
>>>> On Wed, Feb 13, 2013 at 3:04 AM, Maria Stylianou <>wrote:
>>>>> Hey,
>>>>>  I am trying to run the example SimpleMasterComputeVertex, but no
>>>>> matter which Input Format and graph I give, it doesn't work. Each worker
>>>>> gives the error:
>>>>> Caused by: java.lang.*NullPointerException*
>>>>> 	at org.apache.giraph.examples.*SimpleMasterComputeVertex.compute*(*42*)
>>>>> This line 42 is the first line of the compute()
>>>>> public void compute(Iterable<DoubleWritable> messages){
>>>>>  So I guess, the initialization is not done correctly, because the
>>>>> input file does not have the correct format.
>>>>>  Any help would be appreciated,
>>>>> Thanks!
>>>>>  Maria
>>>>> --
>>>>> Maria Stylianou
>>>>> Intern at Telefonica, Barcelona, Spain
>>>>> Master Student of European Master in Distributed Computing<>
>>>>> Universitat Polit├Ęcnica de Catalunya - BarcelonaTech, Barcelona, Spain
>>>>>  KTH Royal Institute of Technology, Stockholm, Sweden

View raw message