incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Jungblut (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-363) Add network condition monitoring function to BSPMaster
Date Mon, 09 May 2011 21:02:03 GMT

    [ https://issues.apache.org/jira/browse/HAMA-363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13030895#comment-13030895
] 

Thomas Jungblut commented on HAMA-363:
--------------------------------------

As far as I know Hadoop only provides some JVM metrics and host metrices. I don't exactly
find the correct source code position, but I think we should implement our own metrics package,
which we can later add to ganglia. This is much more useful. 

We should define things we need to determine whether there are problems or not. 
Something like: "We ping every groom every 5 seconds and check the latency."
This can be easily implemented in BSPMaster.

To measure the IN and OUT rate or other fancy stuff we need something like heartbeat communication
that will transfer the local groom data to the master. 
This should be in the newer versions of Hadoop >0.21 shouldn't it? Don't have the source
codes haging around here.

> Add network condition monitoring function to BSPMaster
> ------------------------------------------------------
>
>                 Key: HAMA-363
>                 URL: https://issues.apache.org/jira/browse/HAMA-363
>             Project: Hama
>          Issue Type: New Feature
>          Components: bsp
>    Affects Versions: 0.3.0
>            Reporter: Edward J. Yoon
>             Fix For: 0.3.0
>
>
> There's few reason why this issue is important. Basically Master server should know the
status of cluster:
>  * to optimize the network usage
>  * to handle network connectivity problems
>  * to handle different network conditions
> And, I would like to see some network usage statistics.
> In this issue, we implement only basic function which is collect network usage.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message