tez-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TEZ-2609) Consider calling TaskReporter...->..heartbeat() when there are no more events to be processed in LogicalIOProcessorRuntimeTask
Date Thu, 09 Jul 2015 14:17:05 GMT

    [ https://issues.apache.org/jira/browse/TEZ-2609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620553#comment-14620553
] 

Jeff Zhang commented on TEZ-2609:
---------------------------------

Maybe we could add some randomness on the heartbeat interval rather than using the same interval
for all the containers. This would help to even the rpc traffic on AM.

   

> Consider calling TaskReporter...->..heartbeat() when there are no more events to be
processed in LogicalIOProcessorRuntimeTask
> ------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: TEZ-2609
>                 URL: https://issues.apache.org/jira/browse/TEZ-2609
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Rajesh Balamohan
>
> Default TEZ_TASK_AM_HEARTBEAT_INTERVAL_MS is around 100 ms.  This works for most of the
usecases. However, for large jobs (10000s of tasks), this can be a problem and have timeout
issues. Setting this to a very large value would degrade the job runtime and lower value can
cause timeout issues for large jobs.
> It might be worth to consider deferring heartbeat() when there are events to be processed
in LogicalIOProcessorRuntimeTask.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message