Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Wed, 11 Mar 2015 04:19:39 +0000 (UTC)
From: "Vinod Kumar Vavilapalli (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12780967.1426017387000.59902.1426047579872@Atlassian.JIRA>
In-Reply-To: <JIRA.12780967.1426017387000@Atlassian.JIRA>
References: <JIRA.12780967.1426017387000@Atlassian.JIRA>
 <JIRA.12780967.1426017387054@arcas>
Subject: [jira] [Commented] (YARN-3332) [Umbrella] Unified Resource
 Statistics Collection per node
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/YARN-3332?page=3Dcom.atlassian.=
jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D14356=
260#comment-14356260 ]=20

Vinod Kumar Vavilapalli commented on YARN-3332:
-----------------------------------------------

Chose the service model because the machine level big picture is fragmented=
 between YARN and HDFS (and HBase etc) - having a lower level common statis=
tics layer is useful.

I anyways needed a service to expose an API for both admins/users as well a=
s external systems beyond HDFS too - I can imagine tools being built on top=
 of this.

That said, it doesn't need to be service or library. I can think of a libra=
ry that wires into the exposed API, though I haven't found uses for that ye=
t.

> [Umbrella] Unified Resource Statistics Collection per node
> ----------------------------------------------------------
>
>                 Key: YARN-3332
>                 URL: https://issues.apache.org/jira/browse/YARN-3332
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Vinod Kumar Vavilapalli
>         Attachments: Design - UnifiedResourceStatisticsCollection.pdf
>
>
> Today in YARN, NodeManager collects statistics like per container resourc=
e usage and overall physical resources available on the machine. Currently =
this is used internally in YARN by the NodeManager for only a limited usage=
: automatically determining the capacity of resources on node and enforcing=
 memory usage to what is reserved per container.
> This proposal is to extend the existing architecture and collect statisti=
cs for usage b=E2=80=8Beyond=E2=80=8B the existing use=C2=ADcases.
> Proposal attached in comments.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)