Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6F43F11325 for ; Tue, 22 Jul 2014 19:25:55 +0000 (UTC) Received: (qmail 36879 invoked by uid 500); 22 Jul 2014 19:25:50 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 36745 invoked by uid 500); 22 Jul 2014 19:25:50 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 36721 invoked by uid 99); 22 Jul 2014 19:25:50 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 22 Jul 2014 19:25:50 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,MANY_SPAN_IN_TEXT,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of ajkarthik@gmail.com designates 209.85.218.48 as permitted sender) Received: from [209.85.218.48] (HELO mail-oi0-f48.google.com) (209.85.218.48) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 22 Jul 2014 19:25:47 +0000 Received: by mail-oi0-f48.google.com with SMTP id h136so81915oig.35 for ; Tue, 22 Jul 2014 12:25:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=LYy7NZwKTENM0F4eY/1thVyjRgebTEt1BiDQwahL2cw=; b=cY9EzO+s7mNZZi3JOzKsmU/CrkEWXAW1cD4G+7Iiq2t6SJBHitBNaJwaf81eTUFxM4 bSLnQOYidL4CuLwpDUB42XcySiZqLISA84AxQFY/T0B00rtfSE9wqnEZPhV/lmlNoYtE vFdU56dAAZWdB9WTPTEycn+a08Z8mQ2CgSedDazjKGXX0qDHKDHQ2tqXQPtlplp2MI7o z8tZAIFz+kvpY8AkCvBgsP+2Y8O+NHH1pzi52Q2iG5YICJkx2g9B263ObidpNSItw5Dz pN74HZREJkdZYzRK2NzAh2YeDYgWNeV25lAccUBHErSJVUgMKD/TwScI72PaYYMnhKq8 A4zg== MIME-Version: 1.0 X-Received: by 10.60.146.198 with SMTP id te6mr132685oeb.46.1406057122500; Tue, 22 Jul 2014 12:25:22 -0700 (PDT) Received: by 10.76.90.194 with HTTP; Tue, 22 Jul 2014 12:25:22 -0700 (PDT) Date: Wed, 23 Jul 2014 00:55:22 +0530 Message-ID: Subject: Configuring the Container Logs From: Jogeshwar Karthik Akundi To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=047d7b5d30b65218cc04fecd2f54 X-Virus-Checked: Checked by ClamAV on apache.org --047d7b5d30b65218cc04fecd2f54 Content-Type: text/plain; charset=UTF-8 Hi, I have been trying to configure the Log output of my actual job container files (appid/containerid/sysout, syslog). 1) I want to disable DEBUG in the container logs. 2) I want to redirect a specific package into another file. I tried the steps described in: http://stackoverflow.com/questions/22918720/custom-log4j-appender-in-hadoop-2 but to no avail. Whatever I do, the DEBUG logs wouldn't suppress and my package logs wouldn't redirect. Any pointers? my container-log4j.properties is as below: #set hadoop log hadoop.root.logger=INFO,CLA # Define the root logger to the system property "hadoop.root.logger". log4j.rootLogger=${hadoop.root.logger}, EventCounter # Logging Threshold log4j.threshold=INFO # # ContainerLog Appender # #Default values yarn.app.mapreduce.container.log.dir=null yarn.app.mapreduce.container.log.filesize=100 log4j.appender.CLA=org.apache.hadoop.yarn.ContainerLogAppender log4j.appender.CLA.threshold=INFO log4j.appender.CLA.containerLogDir=${yarn.app.mapreduce.container.log.dir} log4j.appender.CLA.totalLogFileSize= ${yarn.app.mapreduce.container.log.filesize} log4j.appender.CLA.layout=org.apache.log4j.PatternLayout log4j.appender.CLA.layout.ConversionPattern=%d{ISO8601} %p [%t] %c: %m%n #set my package log redirection here log4j.logger.com.my.own.package=INFO,MyDRFA # # Recipe Daily Rolling File Appender # log4j.appender.MyDRFA=org.apache.log4j.DailyRollingFileAppender log4j.appender.MyDRFA.append=true log4j.appender.MyDRFA.File=/home/kaseya/logs/hadoop/recipe.log log4j.appender.MyDRFA.DatePattern=.yyyy-MM-dd log4j.appender.MyDRFA.layout=org.apache.log4j.PatternLayout log4j.appender.MyDRFA.layout.ConversionPattern=%d{ISO8601} %p [%t] %c: %m%n # # console # Add "console" to rootlogger above if you want to use this # log4j.appender.console=org.apache.log4j.ConsoleAppender log4j.appender.console.target=System.err log4j.appender.console.threshold=INFO log4j.appender.console.layout=org.apache.log4j.PatternLayout log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{2}: %m%n # # Event Counter Appender # Sends counts of logging messages at different severity levels to # Hadoop Metrics. # log4j.appender.EventCounter=org.apache.hadoop.log.metrics.EventCounter log4j.logger.org.apache.hadoop.conf.Configuration=INFO Thanks --047d7b5d30b65218cc04fecd2f54 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi,
I have been trying to configure the Log output of = my actual job container files (appid/containerid/sysout, syslog).

1) I want to disable DEBUG in the container logs.
2) I want to redirect a specific package into another file.

I tried the steps described in:

but to no avail.
Whatever I do, the DEBUG log= s wouldn't suppress and my package logs wouldn't redirect.

Any pointers?

my container-log4j.= properties is as below:


#set hadoop log

hadoop.root.logger=3DINFO,CLA

# Define the root logger to the system property "hadoop.root.logger= ".

log4j.rootLogger=3D${hadoop.root.logger}, EventCounter

# Logging Threshold

log4j.threshold=3DINFO

#

# ContainerLog Appender

#

#Default values

yarn.app.mapreduce.container.log.dir= =3Dnull

yarn.app.mapreduce.container.log.filesize=3D100

log4j.appender.CLA=3Dorg.apache.hadoop.yarn.ContainerLogAppend= er

log4j.appender.CLA.threshold=3D= INFO

log4j.appender.CLA.containerLogDir= =3D${yarn.app.mapreduce.cont= ainer.log.dir}

log4j.appender.CLA.totalLogFileSiz= e=3D${yarn.app.mapreduce.con= tainer.log.filesize}

log4j.appender.CLA.layout=3Dorg.apache.log4j.PatternLayout

log4j.appender.CLA.layout.ConversionPattern=3D%d{ISO8601} %p [%t] %c:= %m%n

#set my package log redirection here

log4j.logger.com.my.own.package=3DINFO,MyDRFA

#

# Recipe Daily Rolling File Appender

#

log4j.appender.MyDRFA=3Dorg.apache.log4j.DailyRollingFileAppen= der

log4j.appender.MyDRFA.append=3D= true

log4j.appender.MyDRFA.File<= span style=3D"color:rgb(0,0,0)">=3D/home/kaseya/logs/hadoop/recipe.l= og

log4j.appender.MyDRFA.DatePattern=3D.yyyy-MM-dd

log4j.appender.MyDRFA.layout=3Dorg.apache.log4j.PatternLayout<= /p>

log4j.appender.MyDRFA.layout.ConversionPattern=3D%d{ISO8601} %p [%t] = %c: %m%n

#

# console

# Add "console" to rootlogger above if you want to use this=C2= =A0

#

log4j.appender.console=3Dorg.apache.log4j.ConsoleAppender

log4j.appender.console.target=3DSystem.err

log4j.appender.console.threshold=3DINFO

log4j.appender.console.layout=3Dorg.apache.log4j.PatternLayout=

log4j.appender.console.layout.ConversionPattern=3D%d{yy/MM/dd HH:mm:s= s} %p %c{2}: %m%n

#

# Event Counter Appender

# Sends counts of logging messages at different severity levels to

# Hadoop Metrics.

#

log4j.appender.EventCounter= =3Dorg.apache.hadoop.log.metrics.Ev= entCounter

log4j.logger.org.apache.hadoop.conf.Configuration=3DINFO


Thanks

--047d7b5d30b65218cc04fecd2f54--