reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Markus Weimer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (REEF-1484) Document IMRU collaboration/data flow
Date Wed, 06 Jul 2016 17:54:11 GMT

    [ https://issues.apache.org/jira/browse/REEF-1484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364743#comment-15364743
] 

Markus Weimer commented on REEF-1484:
-------------------------------------

Thanks for bringing this up! This is probably best captured on the website, not the wiki,
as it is user documentation.We currently confuse the heck out of people by putting some user
docs in the wiki :)

> Document IMRU collaboration/data flow
> -------------------------------------
>
>                 Key: REEF-1484
>                 URL: https://issues.apache.org/jira/browse/REEF-1484
>             Project: REEF
>          Issue Type: Bug
>          Components: Documentation
>            Reporter: Andrey
>
> need to document IMRU components interaction. 
> perhaps small wiki with collaboration and data flow diagrams.
> IMRU main flow fairly obvious, but following things required some debugging/reverse engineering:
> - input data is not used unless constructor of MapFunction defines it as parameter
> - how to configure input data set
> - Update function returns iEnumerable of results as generic case. it's up to algorithm
to decide which result should be included into the list: it can decide include result from
last iteration only
> - Map input/output codecs are required in order to serialize/deserialize data across
nodes.
> - Optional Map input/output data converters can be used by algorithm developer to improve
performance of distributed calculations. For instance send subset of dimensions to a node.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message