giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jan van der Lugt (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GIRAPH-192) Move aggregators to a seperate sub-package
Date Tue, 22 May 2012 16:44:40 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281095#comment-13281095
] 

Jan van der Lugt commented on GIRAPH-192:
-----------------------------------------

The aggregators I provided in the patch are the same as the ones in the .example package,
just slightly modified by replacing the types or operations (generic aggregators are sadly
not a possibility). The product aggregators, for example, are the same as the sum aggregators,
except with += replaced by *=. You are right in mentioning that aggregators are part of the
Google Pregel BSP design, but so are combiners. The problem with combiners is that they are
dependent on specific message formats, while aggregators work directly on values. This is
why I believe Giraph should provide reusable aggregators, but not combiners.

Regarding your first issue with aggregators: I implemented Pagerank with an aggregator that
is used to calculate the global sum of errors. The values calculated by this aggregator were
exactly the same as calculated by another algorithm (built on a different, in-memory, single
machine graph framework). This suggests to me that they are, at least in my case, working
correctly. You might want to check your code again, is the reset performed in the preSuperstep()
function? The mailing list might be a better place to discuss this than this JIRA, however.
                
> Move aggregators to a seperate sub-package
> ------------------------------------------
>
>                 Key: GIRAPH-192
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-192
>             Project: Giraph
>          Issue Type: Improvement
>          Components: examples
>            Reporter: Jan van der Lugt
>            Priority: Minor
>         Attachments: GIRAPH-192.patch
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> Since aggregators will be re-used throughout many projects and algorithms, it makes sense
to implement the most common ones in a separate sub-package. This will reduce the time required
for users when they implement their projects based on Giraph, because the required aggregators
are already in place. I implemented the following ones:
> for int/long/float/double: min, max, product, sum, overwrite
> for boolean: and, or, overwrite
> Most of them speak for themselves, except for the overwrite one. This aggregator simply
overwrites the stored value when a new value is aggregated. This is useful when one node is
in some way a master node (for example a source node in an routing algorithm), and this node
wants to broadcast a value to all other nodes.
> Attached is a patch against trunk implementing the aggregators and patching some existing
files so they use the .aggregators package instead of the .examples one.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message