hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yufei Gu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-6546) SLS is slow while loading 10k queues
Date Wed, 11 Oct 2017 23:15:03 GMT

     [ https://issues.apache.org/jira/browse/YARN-6546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Yufei Gu updated YARN-6546:
    Attachment: YARN-6546.002.patch

Thanks for the review, [~miklos.szegedi@cloudera.com]. Uploaded patch v2 for your comments.
In patch v2:
- SLS tracks a queue when there is an app in it. 
- Remove {{untrackQueue()}} since it is broken and not used.

> SLS is slow while loading 10k queues
> ------------------------------------
>                 Key: YARN-6546
>                 URL: https://issues.apache.org/jira/browse/YARN-6546
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: scheduler-load-simulator
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Yufei Gu
>            Assignee: Yufei Gu
>         Attachments: Desktop.png, YARN-6546.001.patch, YARN-6546.002.patch
> It takes a long time (more than 10 minutes) to load 10k queues in SLS. The problem should
be in {{com.codahale.metrics.CsvReporter}} based on the result from profiler. SLS creates
14 .csv files for each leaf queue, and update them constantly during execution. It is not
necessary to log information for inactive queues. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message