giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eli Reisman <apache.mail...@gmail.com>
Subject Re: InputFormat for the example SimpleMasterComputeVertex
Date Sat, 02 Mar 2013 22:01:03 GMT
Its fantastic thanks again! We need more wiki pages like that!!! This will
save so many emails on the list too! I need to (and will) update the graph
representations page to say we support multigraph, edge and vertex IO, etc.
soon. A lot of our wiki info is old enough to be confusing people now.

On Sat, Mar 2, 2013 at 1:57 PM, Eli Reisman <apache.mailbox@gmail.com>wrote:

> Thanks for doing this Maja!
>
>
> On Thu, Feb 21, 2013 at 5:12 PM, Maja Kabiljo <majakabiljo@fb.com> wrote:
>
>>  I wrote up a basic info about aggregators:
>> https://cwiki.apache.org/confluence/display/GIRAPH/Aggregators. Please
>> take a look, and let me know if something needs to be changed / improved.
>>
>>   From: Eli Reisman <apache.mailbox@gmail.com>
>> Reply-To: "user@giraph.apache.org" <user@giraph.apache.org>
>> Date: Thursday, February 21, 2013 4:21 PM
>>
>> To: "user@giraph.apache.org" <user@giraph.apache.org>
>> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>>
>>  That sounds great to me, maybe just a mention in the wiki that the two
>> functionalities are tied together will help the idea click for people.
>> Either way this will be a big help I think.
>>
>>
>> On Thu, Feb 21, 2013 at 3:24 PM, Maja Kabiljo <majakabiljo@fb.com> wrote:
>>
>>>  Eli, that's an interesting idea, we could have some class which user
>>> extends and which is there only for aggregator registration. Sometimes we
>>> want to register some aggregators later on during the computation, so we
>>> need to keep allowing registration from masterCompute too.
>>>
>>>  But I think for users the biggest problem is to realize that they have
>>> to extend and set MasterCompute/this new class in order to use aggregators.
>>> Currently, if user tries to aggregate a value to unregistered aggregator he
>>> will get an exception, but if he tries to get the value of unregistered
>>> aggregator he will just get null. So maybe adding a warning message in that
>>> case, together with a wiki page, might be enough? What do you think?
>>>
>>>   From: Eli Reisman <apache.mailbox@gmail.com>
>>> Reply-To: "user@giraph.apache.org" <user@giraph.apache.org>
>>>  Date: Thursday, February 21, 2013 10:25 AM
>>>
>>> To: "user@giraph.apache.org" <user@giraph.apache.org>
>>> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>>>
>>>  Thanks for the explanation, that makes sense. I would love to see a
>>> wiki page at some point, you have so much knowledge of this piece of Giraph
>>> from all your dev work on it and have also the additional bonus of
>>> experience running big cluster jobs using these features so you have a lot
>>> of insight to share.
>>>
>>>  Would there be any point to a future JIRA to break out the aggregator
>>> registration from the master compute stuff, at least from the user's view?
>>> Or is it not that confusing once you've used them a bit?
>>>
>>>
>>> On Thu, Feb 14, 2013 at 4:52 PM, Maja Kabiljo <majakabiljo@fb.com>wrote:
>>>
>>>>  Progressable exception can be caused by many different reasons (it's
>>>> totally unrelated to aggregators), and when looking at which exception it's
>>>> caused by users should get better sense about what's going on.
>>>> What you are suggesting about providing default master compute is not
>>>> doable, since the part which needs to be done there is aggregator
>>>> registration. We can't know what kind of aggregators (names and types) an
>>>> application needs.
>>>> I remember I was talking about writing a short tutorial for aggregators
>>>> long time ago, sorry for not doing that, will try to get to it soon.
>>>>
>>>>   From: Eli Reisman <apache.mailbox@gmail.com>
>>>> Reply-To: "user@giraph.apache.org" <user@giraph.apache.org>
>>>> Date: Thursday, February 14, 2013 2:23 PM
>>>>
>>>> To: "user@giraph.apache.org" <user@giraph.apache.org>
>>>> Subject: Re: InputFormat for the example SimpleMasterComputeVertex
>>>>
>>>>   Other folks on the list are also having this problem with the
>>>> progressable utile exception & job failures. I don't know much about
master
>>>> compute usage but if it is needed to make the aggregators work, maybe we
>>>> should have a default dummy class that just handles aggregators if no other
>>>> master compute is specified? Or a wiki page? The progressable error message
>>>> does not lead us to this conclusion directly.
>>>>
>>>> On Wed, Feb 13, 2013 at 3:04 AM, Maria Stylianou <marsty5@gmail.com>wrote:
>>>>
>>>>> Hey,
>>>>>
>>>>>  I am trying to run the example SimpleMasterComputeVertex, but no
>>>>> matter which Input Format and graph I give, it doesn't work. Each worker
>>>>> gives the error:
>>>>>
>>>>> Caused by: java.lang.*NullPointerException*
>>>>> 	at org.apache.giraph.examples.*SimpleMasterComputeVertex.compute*(SimpleMasterComputeVertex.java:*42*)
>>>>>
>>>>> This line 42 is the first line of the compute()
>>>>> public void compute(Iterable<DoubleWritable> messages){
>>>>>
>>>>>  So I guess, the initialization is not done correctly, because the
>>>>> input file does not have the correct format.
>>>>>
>>>>>  Any help would be appreciated,
>>>>> Thanks!
>>>>>  Maria
>>>>> --
>>>>> Maria Stylianou
>>>>> Intern at Telefonica, Barcelona, Spain
>>>>> Master Student of European Master in Distributed Computing<http://www.kth.se/en/studies/programmes/master/em/emdc>
>>>>> Universitat Polit├Ęcnica de Catalunya - BarcelonaTech, Barcelona, Spain
>>>>>  KTH Royal Institute of Technology, Stockholm, Sweden
>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message