hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sri Ramadasu (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1900) the heartbeat and task event queries interval should be set dynamically by the JobTracker
Date Thu, 18 Oct 2007 05:46:51 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12535840
] 

Amareshwari Sri Ramadasu commented on HADOOP-1900:
--------------------------------------------------

bq. I wonder if instead we should just make it clusterSize/50+1? That way, small clusters
will get a heartbeat of just one second, which should make them more responsive.

Agreed.

bq. Now, MapEventsFetcherThread polls jobtracker for completed map tasks for every 5 secs
(MIN_POLL_INTERVAL). Shall we change polling interval also in the similar fashion as heartbeat
interval? But, here some reduce tasks could be idle for longer time.

We can have MapEventsFetcherThread polling jobtracker for completed map tasks  in the similar
fashion as heartbeat interval with MIN_POLL_INTERVAL=5secs and MIN_POLL_INTERVAL_MAX=30secs.
 And Whenever tasktracker finds there are no mapevents and reduce task is waiting for the
map events, it will wakeup the thread to fetch map events from job tracker.

> the heartbeat and task event queries interval should be set dynamically by the JobTracker
> -----------------------------------------------------------------------------------------
>
>                 Key: HADOOP-1900
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1900
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Owen O'Malley
>            Assignee: Owen O'Malley
>
> The JobTracker should scale the intervals that the TaskTrackers use to contact it dynamically,
based on how the busy it is and the size of the cluster.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message