hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maria <linanmengxia...@126.com>
Subject Fw:Re:How to add custom field to hadoop MR task log?
Date Fri, 04 Nov 2016 09:01:16 GMT

I know that, A simple way is to write "<custom ID> " to every LOG.info()/LOG.warn()....like

logger.info(ID + " start map logic");
BUT,every LOG info has to add "ID" is not wise.   
Or else, can someone know how to modify the mapreduce task ConversionPattern configuration?
I tried to modify "RFA" Appender to this:
# Pattern format: Date LogLevel LoggerName LogMessage
log4j.appender.RFA.layout.ConversionPattern=%d{ISO8601} %p %c: %m%n
# Debugging Pattern format
#log4j.appender.RFA.layout.ConversionPattern=%d{ISO8601} %-5p %l - [ID: %X{ID}]  %m%n
It does not work.

At 2016-11-04 11:26:57, "Maria" <linanmengxiaoai@126.com> wrote:
>Hi, dear developers,
>I'm trying to reconfig $HADOOP/etc.hadoop/log4j.properties, 
>I want to add an <custom ID> to mapreduce log before LOGmessage. Like this:
>"ID:234521 start map logic" 
>My steps as follow:
>(1)In my Mapper Class:
>static Logger logger = LoggerFactory.getLogger(Mapper.class);
>public void map(Object key, Text value, Context context) throws IOException, InterruptedException
>    MDC.put("ID", "operatorID");
>    logger.info("start map logic");
>     StringTokenizer itr = new StringTokenizer(value.toString());
>     while (itr.hasMoreTokens()) {
>       word.set(itr.nextToken());
>       context.write(word, one);
>     }
>   }
> }
>(2)config $HADOOP/etc.hadoop/log4j.properties
>log4j.appender.TLA.layout.ConversionPattern=%l  %p %c: ID:[%X{ID}]  %m%n 
>BUT it does not work. and because use slf4j API, so I don't know how to get Appenders.
>I am desperately in need。。
>Any help would be highly appreciated
>To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
>For additional commands, e-mail: user-help@hadoop.apache.org

To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
For additional commands, e-mail: user-help@hadoop.apache.org
View raw message