hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Jungblut (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-567) BSPPeer should provide means for chaining supersteps to share data among them.
Date Thu, 03 May 2012 13:36:48 GMT

    [ https://issues.apache.org/jira/browse/HAMA-567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267427#comment-13267427
] 

Thomas Jungblut commented on HAMA-567:
--------------------------------------

Hey Suraj, please give me a bit of time to sort the things out we need. I wanted to give a
shot at a prototype for the other issue first, we can then discuss which parts can be shared
through both cases.

I wanted to work through some caching solutions like EHCache or DirectMemory and see how they
do that.
For us I guess retrieval by int should be enough, so we can use Trove library to have low
overhead hashmaps.

However, there are cases where someone wants to build their own key objects, therefore a generic
solution wouldn't be so wrong. 
So there must be some polymorphism in this case. 

BTW I think your mapper can be drastically improved ;) Maybe we can have a talk later.
                
> BSPPeer should provide means for chaining supersteps to share data among them.
> ------------------------------------------------------------------------------
>
>                 Key: HAMA-567
>                 URL: https://issues.apache.org/jira/browse/HAMA-567
>             Project: Hama
>          Issue Type: Improvement
>          Components: bsp core
>    Affects Versions: 0.6.0
>            Reporter: Suraj Menon
>             Fix For: 0.6.0
>
>         Attachments: Mapper.java
>
>
> In most scenarios, a superstep would need certain values or objects that were computed
in the previous superstep. When using the chaining Superstep design to implement BSP algorithms,
this gets a little ugly/difficult to implement. BSPPeer should provide means (preferably a
map<String,Object>) so that the next Superstep can ask for the values in previous superstep
using String token to query the map. Also, this map could be checkpointed periodically in
the background so that we can completely recover the state of a task after failure. The BSPPeer
object should have a dedicated get and set function for updating values in the peer.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message