hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1383) Remove node updates from the Fair Scheduler event log
Date Thu, 14 Nov 2013 12:09:21 GMT

    [ https://issues.apache.org/jira/browse/YARN-1383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13822356#comment-13822356

Karthik Kambatla commented on YARN-1383:

Sorry for the delay, somehow skipped my mind.

I remember using the FS event log when debugging at least 4 production issues. On one such
occasion, when jobs/tasks were surprisingly slow, I remember using the FS event log to check
what nodes the tasks were being assigned to and if those nodes were heartbeating at all. The
issue turned out to be node-specific. It might be possible to arrive at the same conclusion
through other means, FS event log makes it super-easy to debug these situations. 

I completely understand it is too expensive to log these messages all the time. However, it
would be nice to at least log these at a trace level. When debugging an issue requiring this,
we can always turn on trace-level logging.

> Remove node updates from the Fair Scheduler event log
> -----------------------------------------------------
>                 Key: YARN-1383
>                 URL: https://issues.apache.org/jira/browse/YARN-1383
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: scheduler
>    Affects Versions: 2.2.0
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
>         Attachments: YARN-1383.patch
> Writing out a line whenever a node heartbeats is not useful and just too much.

This message was sent by Atlassian JIRA

View raw message