hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suraj Menon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-742) Implement of Hama RPC
Date Thu, 07 Mar 2013 21:56:13 GMT

    [ https://issues.apache.org/jira/browse/HAMA-742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13596435#comment-13596435
] 

Suraj Menon commented on HAMA-742:
----------------------------------

16K was set because I didn't see any performance improvement since 16K with an 8k intermediate
buffer. Remember Spilling Buffer is a circular array of configurable size that contains configurable
sized buffers. I chose the lowest because, the messaging memory is multiplied by the number
of peers a task is going to communicate to. If we go in MBs for each peer, the memory usage
is too high. 

Unfortunately, my code is across projects with some passing and some failing unit tests. I
shall upload few building blocks by this weekend.
                
> Implement of Hama RPC 
> ----------------------
>
>                 Key: HAMA-742
>                 URL: https://issues.apache.org/jira/browse/HAMA-742
>             Project: Hama
>          Issue Type: Sub-task
>            Reporter: Edward J. Yoon
>             Fix For: 0.6.1
>
>
> To solve HDFS 2.0 compatibility issue, we have to change a lot of codes for Hadoop 2.0
RPC, moreover, yarn RPC doesn't support asynchronous call directly.
> Ultimately, we can pursue the performance and integrate more easily with hadoop multi-versions
by having our own RPC.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message