hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From templ...@apache.org
Subject hadoop git commit: YARN-6453. fairscheduler-statedump.log gets generated regardless of service (Contributed by Yufei Gu via Daniel Templeton)
Date Thu, 20 Apr 2017 21:15:15 GMT
Repository: hadoop
Updated Branches:
  refs/heads/branch-2 1bbba00a2 -> d77137a03


YARN-6453. fairscheduler-statedump.log gets generated regardless of service
(Contributed by Yufei Gu via Daniel Templeton)

(cherry picked from commit e74ca7216aecc998d277e5ca1f4d0abae43c7e31)


Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/d77137a0
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/d77137a0
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/d77137a0

Branch: refs/heads/branch-2
Commit: d77137a03f89faf43cdf4702e9d1c58febdf130a
Parents: 1bbba00
Author: Daniel Templeton <templedf@apache.org>
Authored: Thu Apr 20 14:11:01 2017 -0700
Committer: Daniel Templeton <templedf@apache.org>
Committed: Thu Apr 20 14:14:38 2017 -0700

----------------------------------------------------------------------
 .../src/main/conf/log4j.properties              | 22 ++++++++++++--------
 .../src/site/markdown/FairScheduler.md          |  6 ++++++
 2 files changed, 19 insertions(+), 9 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hadoop/blob/d77137a0/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
----------------------------------------------------------------------
diff --git a/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties b/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
index b83eb6f..89b7d41 100644
--- a/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
+++ b/hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
@@ -322,12 +322,16 @@ log4j.appender.EWMA.cleanupInterval=${yarn.ewma.cleanupInterval}
 log4j.appender.EWMA.messageAgeLimitSeconds=${yarn.ewma.messageAgeLimitSeconds}
 log4j.appender.EWMA.maxUniqueMessages=${yarn.ewma.maxUniqueMessages}
 
-# Fair scheduler requests log on state dump
-log4j.logger.org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.statedump=DEBUG,FSLOGGER
-log4j.additivity.org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.statedump=false
-log4j.appender.FSLOGGER=org.apache.log4j.RollingFileAppender
-log4j.appender.FSLOGGER.File=${hadoop.log.dir}/fairscheduler-statedump.log
-log4j.appender.FSLOGGER.layout=org.apache.log4j.PatternLayout
-log4j.appender.FSLOGGER.layout.ConversionPattern=%d{ISO8601} %p %c: %m%n
-log4j.appender.FSLOGGER.MaxFileSize=${hadoop.log.maxfilesize}
-log4j.appender.FSLOGGER.MaxBackupIndex=${hadoop.log.maxbackupindex}
\ No newline at end of file
+#
+# Fair scheduler state dump
+#
+# Use following logger to dump the state to a separate file
+
+#log4j.logger.org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.statedump=DEBUG,FSSTATEDUMP
+#log4j.additivity.org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.statedump=false
+#log4j.appender.FSSTATEDUMP=org.apache.log4j.RollingFileAppender
+#log4j.appender.FSSTATEDUMP.File=${hadoop.log.dir}/fairscheduler-statedump.log
+#log4j.appender.FSSTATEDUMP.layout=org.apache.log4j.PatternLayout
+#log4j.appender.FSSTATEDUMP.layout.ConversionPattern=%d{ISO8601} %p %c: %m%n
+#log4j.appender.FSSTATEDUMP.MaxFileSize=${hadoop.log.maxfilesize}
+#log4j.appender.FSSTATEDUMP.MaxBackupIndex=${hadoop.log.maxbackupindex}

http://git-wip-us.apache.org/repos/asf/hadoop/blob/d77137a0/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/FairScheduler.md
----------------------------------------------------------------------
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/FairScheduler.md
b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/FairScheduler.md
index a3cb900..970058d 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/FairScheduler.md
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/FairScheduler.md
@@ -228,3 +228,9 @@ The following fields can be seen for each queue on the web interface:
 The Fair Scheduler supports moving a running application to a different queue. This can be
useful for moving an important application to a higher priority queue, or for moving an unimportant
application to a lower priority queue. Apps can be moved by running `yarn application -movetoqueue
appID -queue targetQueueName`.
 
 When an application is moved to a queue, its existing allocations become counted with the
new queue's allocations instead of the old for purposes of determining fairness. An attempt
to move an application to a queue will fail if the addition of the app's resources to that
queue would violate the its maxRunningApps or maxResources constraints.
+
+###Dumping Fair Scheduler state
+
+Fair Scheduler is able to dump its state periodically. It is disabled by default. The administrator
can enable it by setting org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler
logging level to DEBUG.
+
+Fair Scheduler logs go to the Resource Manager log file by default. Fair Scheduler state
dumps can potentially generate large amount of log data. Uncomment the "Fair scheduler state
dump" section in log4j.properties to dump the state into a separate file.


---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org


Mime
View raw message