giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jan van der Lugt (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GIRAPH-259) TestBspBasic.testBspPageRank is broken
Date Fri, 03 Aug 2012 05:00:04 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427834#comment-13427834
] 

Jan van der Lugt commented on GIRAPH-259:
-----------------------------------------

Ah, now I understand. Yes, this is an interesting issue. I would argue for an approach where
the master can't add to a SumAggregator (so it cannot use the aggregate() call), but only
use setAggregatedValue(). The vertices, in turn, should only use aggregate() and never setAggregatedValue(),
since the latter would cause non-deterministic executions. I guess it's better to guard people
from making these mistakes. Of course the master could still read the aggregated value, add
something to it and set that as the new value, but that simply counts as setting a new value.
In this way, the aggregator can reset to the default value if the master does not set a new
value. This way we circumvent the 15/16 issue, since the master has to explicitly set the
value every step. We would have to have 2 aggregator interfaces for this... Does this all
make sense?

I still argue for an API without a WorkerContext handling aggregators, since I believe all
of this can be handled by a MasterCompute at a much nicer level of abstraction. Only registerAggregator()
and useAggregator() need to be handled by the WorkerContext, but isn't it easier to just let
all workers register and use all aggregators by default? I don't think this really adds much
of an overhead, since the state of each aggregator is really small (usually just an object
with one primitive value). Would like to hear your thoughts on this.
                
> TestBspBasic.testBspPageRank is broken
> --------------------------------------
>
>                 Key: GIRAPH-259
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-259
>             Project: Giraph
>          Issue Type: Bug
>            Reporter: Maja Kabiljo
>            Assignee: Maja Kabiljo
>         Attachments: GIRAPH-259-1.patch, GIRAPH-259-2.patch, GIRAPH-259-3.patch
>
>
> Test crashes on line 152 in class SimplePageRankVertex in distributed mode.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message