hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Jungblut (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-559) Add a caching message queue
Date Thu, 20 Sep 2012 08:29:08 GMT

    [ https://issues.apache.org/jira/browse/HAMA-559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459448#comment-13459448
] 

Thomas Jungblut commented on HAMA-559:
--------------------------------------

The question is what we really need. A queue normally just get read sequentially in our case,
so it would indeed by advantageous to only spill to disk if it exceeds a certain limit (64m
chunk or something other configured).

Ehcache is about caching something from a slow device, which does not give us benefits, because
messages from usercode reside in memory first.
This random access vs. sequential access, I believe we only have the latter. Feel free to
correct me with the usecases of ehcache, you're working on it ;)
                
> Add a caching message queue
> ---------------------------
>
>                 Key: HAMA-559
>                 URL: https://issues.apache.org/jira/browse/HAMA-559
>             Project: Hama
>          Issue Type: New Feature
>          Components: bsp core
>    Affects Versions: 0.5.0
>            Reporter: Thomas Jungblut
>            Priority: Minor
>             Fix For: 0.6.0
>
>
> After HAMA-521 is done, we can add a caching queue which just holds the messages in RAM
that fit into the heap space. The rest can be flushed to disk.
> We may call this a HybridQueue or something like that.
> The benefits should be that we don't have to flush to disk so often and get faster. However
we may have more GC so it is always overall faster.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message