hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Vasudev (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-3294) Allow dumping of Capacity Scheduler debug logs via web UI for a fixed time period
Date Wed, 25 Mar 2015 15:47:54 GMT

     [ https://issues.apache.org/jira/browse/YARN-3294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Varun Vasudev updated YARN-3294:
--------------------------------
    Attachment: apache-yarn-3294.4.patch

Uploaded a new patch to address the feedback by [~jianhe].

{quote}
    how about passing the ‘logHierarchy’ as a parameter too ? so that the web service
also work for user-provided class name, may be also take in logLevel and dumpLocation as a
parameter too ?
{quote}

I would prefer not to have those as params. One of the concerns was that users will dump log
files with different filenames which won't get rotated and use up disk space. Similarly with
the hierarchy - we just end up with a slightly different version of the applet we already
have.

bq. question - wondering whether we need the “@QueryParam” for the 'time' parameter, as
other some other places in the same file do, e.g. getApps;

Good point. I've marked it as a FormParam.

bq.    if no permission for RM to write logs, will the webService throw error ?
Yes, I've updated the Javascript code to throw up an alert letting you know there was an error
with the request.

bq.    new File(System.getProperty("hadoop.log.dir"), targetFilename);, should this be “yarn.log.dir”
?

Fixed.

bq.    return "Capacity scheduler logs are being created."; it should probably return JAXB
formatted response. The response seems broken if I directly access the web service from my
browser

The method is a POST method, with no GET method - that's probably why you got the error. From
my machine using curl,
{noformat}
curl -i -X POST -d "time=60" http://localhost:8088/ws/v1/cluster/scheduler/logs
HTTP/1.1 200 OK
Cache-Control: no-cache
Expires: Wed, 25 Mar 2015 02:07:33 GMT
Date: Wed, 25 Mar 2015 02:07:33 GMT
Pragma: no-cache
Expires: Wed, 25 Mar 2015 02:07:33 GMT
Date: Wed, 25 Mar 2015 02:07:33 GMT
Pragma: no-cache
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(6.1.26)

Capacity scheduler logs are being created.
{noformat}

bq.    AdHocLogDumper#appenderLevels - entry is never removed from the map, is this excepted?

I was creating a new appenderLevels hash every time the dumpLogs function is called, changed
to a clear now.
{noformat}
appenderLevels.clear();
{noformat}

bq.    may be mark private/unstable for the AdHocLogDumper for now

Fixed.


> Allow dumping of Capacity Scheduler debug logs via web UI for a fixed time period
> ---------------------------------------------------------------------------------
>
>                 Key: YARN-3294
>                 URL: https://issues.apache.org/jira/browse/YARN-3294
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacityscheduler
>            Reporter: Varun Vasudev
>            Assignee: Varun Vasudev
>         Attachments: Screen Shot 2015-03-12 at 8.51.25 PM.png, apache-yarn-3294.0.patch,
apache-yarn-3294.1.patch, apache-yarn-3294.2.patch, apache-yarn-3294.3.patch, apache-yarn-3294.4.patch
>
>
> It would be nice to have a button on the web UI that would allow dumping of debug logs
for just the capacity scheduler for a fixed period of time(1 min, 5 min or so) in a separate
log file. It would be useful when debugging scheduler behavior without affecting the rest
of the resourcemanager.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message