hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Vasudev (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-3348) Add a 'yarn top' tool to help understand cluster usage
Date Wed, 08 Apr 2015 13:40:13 GMT

     [ https://issues.apache.org/jira/browse/YARN-3348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Varun Vasudev updated YARN-3348:
--------------------------------
    Attachment: apache-yarn-3348.2.patch

Thanks for the reviews [~aw] and [~jianhe].

bq. Why are we doing this manipulation here and not in the Java code?
I get different values when I run the command in the yarn script vs spawn it via Java. From
Java, I get lower values - 80x24, whereas the yarn script gives me 204x44.

bq. backticks are antiquated in modern bash. Use $() construction
Fixed.

bq. What happens if tput gives you zero or an error because you are on a non-addressable terminal?
(You can generally simulate this by unset TERM or equivalent env var)
Thank you for pointing this out. I hadn't considered it. I've added additional checks in the
script. If the values can't be determined either by the script or by the Java code, it sets
it to 80x24.

bq.  “Unable to fetach cluster metrics” - typo
Fixed.

bq. exceeding 80 Column limit,
Fixed.

bq. the -rows, -cols options seems not having effect on my screen when I tried it, could you
double check ?
I found an issue with cols option which I've fixed. Can you please try it again?

bq. the ‘yarn top’ output is repeatedly showing up on terminal every $delay seconds. it’ll
be better to only show that only once.
I didn't understand this - do you mean that it shouldn't auto-refresh?

bq. Does the patch only show root queue info ? should we show all queues info ?
Queues can be specified as a comma seperated string using the "-queues" option. By default,
it shows information for the root queue.

bq. “F + Enter : Select sort field” ; may be use ’S’ for sorting ?
Fixed.

bq. “Memory seconds(in GBseconds” - missing “)”
Fixed

{quote}
It seems a bit odd to have this method in a public API record. Do you know why hashcode is
not correct without this method ? Or we can just type cast it to GetApplicationsRequestPBImpl
and use the method from there.

    // need this otherwise the hashcode doesn't get generated correctly
request.initAllFields();

    for the caching in ClientRMService. Do you think we can do the cache on client side ?
that’ll save RPCs, especially if we have many top commands running on client side.
{quote}
Fixed. Moved the cache to YarnClientImpl where the hashcode doesn't show up. As to why it
wasn't correct - I suspect it might be to do with lazy initialization but I'm not sure.

> Add a 'yarn top' tool to help understand cluster usage
> ------------------------------------------------------
>
>                 Key: YARN-3348
>                 URL: https://issues.apache.org/jira/browse/YARN-3348
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: resourcemanager
>            Reporter: Varun Vasudev
>            Assignee: Varun Vasudev
>         Attachments: apache-yarn-3348.0.patch, apache-yarn-3348.1.patch, apache-yarn-3348.2.patch
>
>
> It would be helpful to have a 'yarn top' tool that would allow administrators to understand
which apps are consuming resources.
> Ideally the tool would allow you to filter by queue, user, maybe labels, etc and show
you statistics on container allocation across the cluster to find out which apps are consuming
the most resources on the cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message