hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6546) SLS is slow while loading 10k queues
Date Tue, 17 Oct 2017 23:29:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16208593#comment-16208593
] 

Hudson commented on YARN-6546:
------------------------------

SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13100 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13100/])
YARN-6546. SLS is slow while loading 10k queues. (Yufei Gu via Haibo (haibochen: rev 46eb1033a86ca53e7b94202567aef8af2417bdf8)
* (edit) hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/FifoSchedulerMetrics.java
* (edit) hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/SLSRunner.java
* (edit) hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SLSCapacityScheduler.java
* (edit) hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/FairSchedulerMetrics.java
* (edit) hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SchedulerMetrics.java
* (edit) hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/CapacitySchedulerMetrics.java
* (edit) hadoop-tools/hadoop-sls/src/main/java/org/apache/hadoop/yarn/sls/scheduler/SLSFairScheduler.java


> SLS is slow while loading 10k queues
> ------------------------------------
>
>                 Key: YARN-6546
>                 URL: https://issues.apache.org/jira/browse/YARN-6546
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: scheduler-load-simulator
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Yufei Gu
>            Assignee: Yufei Gu
>             Fix For: 3.0.0
>
>         Attachments: Desktop.png, YARN-6546.001.patch, YARN-6546.002.patch
>
>
> It takes a long time (more than 10 minutes) to load 10k queues in SLS. The problem should
be in {{com.codahale.metrics.CsvReporter}} based on the result from profiler. SLS creates
14 .csv files for each leaf queue, and update them constantly during execution. It is not
necessary to log information for inactive queues. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message