Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Wed, 9 Sep 2015 17:51:46 +0000 (UTC)
From: "Sunil G (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12859676.1440698597000.292842.1441821106687@Atlassian.JIRA>
In-Reply-To: <JIRA.12859676.1440698597000@Atlassian.JIRA>
References: <JIRA.12859676.1440698597000@Atlassian.JIRA>
 <JIRA.12859676.1440698597314@arcas>
Subject: [jira] [Commented] (YARN-4091) Improvement: Introduce more
 debug/diagnostics information to detail out scheduler activity
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/YARN-4091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14737291#comment-14737291 ] 

Sunil G commented on YARN-4091:
-------------------------------

Thank you [~leftnoteasy]
I understood the scenario mentioned by you. Yes, such cases are not handled in earlier design. As you suggested, if we keep hierarchical structured debug information starting from a heartbeat, and also keeping the assignment order per-application, we can get these information also.

However, my doubt is , we cannot do this for each heartbeat. If we want to do a specific heartbeat for a specific node, we need input from external way. Such a command or REST query etc. 

So I feel we can have a generalized REST query which can take application or queue or node as input. And for some moment, Scheduler can fetch information (debug) in human readable format, it will satisfy all cases. Thoughts?

> Improvement: Introduce more debug/diagnostics information to detail out scheduler activity
> ------------------------------------------------------------------------------------------
>
>                 Key: YARN-4091
>                 URL: https://issues.apache.org/jira/browse/YARN-4091
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacity scheduler, resourcemanager
>    Affects Versions: 2.7.0
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: Improvement on debugdiagnostic information - YARN.pdf
>
>
> As schedulers are improved with various new capabilities, more configurations which tunes the schedulers starts to take actions such as limit assigning containers to an application, or introduce delay to allocate container etc. 
> There are no clear information passed down from scheduler to outerworld under these various scenarios. This makes debugging very tougher.
> This ticket is an effort to introduce more defined states on various parts in scheduler where it skips/rejects container assignment, activate application etc. Such information will help user to know whats happening in scheduler.
> Attaching a short proposal for initial discussion. We would like to improve on this as we discuss.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)