hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maria <linanmengxia...@126.com>
Subject How to add custom field to hadoop MR task log?
Date Fri, 04 Nov 2016 03:26:57 GMT

Hi, dear developers,

I'm trying to reconfig $HADOOP/etc.hadoop/log4j.properties, 
I want to add an <custom ID> to mapreduce log before LOGmessage. Like this:
"ID:234521 start map logic" 

My steps as follow:
(1)In my Mapper Class:

static Logger logger = LoggerFactory.getLogger(Mapper.class);
....

public void map(Object key, Text value, Context context) throws IOException, InterruptedException
{

    MDC.put("ID", "operatorID");
    logger.info("start map logic");

     StringTokenizer itr = new StringTokenizer(value.toString());
     while (itr.hasMoreTokens()) {
       word.set(itr.nextToken());
       context.write(word, one);
     }
   }
 }
(2)config $HADOOP/etc.hadoop/log4j.properties

log4j.appender.TLA=org.apache.hadoop.mapred.TaskLogAppender 
log4j.appender.TLA.taskId=${hadoop.tasklog.taskid} 
log4j.appender.TLA.isCleanup=${hadoop.tasklog.iscleanup} 
log4j.appender.TLA.totalLogFileSize=${hadoop.tasklog.totalLogFileSize} 
   
log4j.appender.TLA.layout=org.apache.log4j.PatternLayout 
log4j.appender.TLA.layout.ConversionPattern=%l  %p %c: ID:[%X{ID}]  %m%n 


BUT it does not work. and because use slf4j API, so I don't know how to get Appenders.

I am desperately in need。。
Any help would be highly appreciated
Mime
View raw message