flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From khadar basha <khadar...@gmail.com>
Subject Re: Log4JAppender Ignoring the Logging Pattern. Getting only description.
Date Fri, 27 Jul 2012 13:58:19 GMT
FlumeLog4jAvroAppender is available as part of 0.94. But its not available
in 1.2.*Log4jAppender *is from flume-ng-log4jappender-1.2.0.jar

I think it is replace as part of 1.2.

On Fri, Jul 27, 2012 at 6:31 PM, Ralph Goers <rgoers@apache.org> wrote:

> You might consider looking at Log4j 2. It has a Flume Appender that
> records the whole formatted log message in the body. In addition, it will
> record the MDC fields as well.
>
> Sent from my iPad
>
> On Jul 27, 2012, at 5:49 AM, khadar basha <khadarskb@gmail.com> wrote:
>
> Hi
>
> I am using Flume1.2. Using avro source and hdfs sink. Sending message
> using the Logger channel from application server. For that i am using the
> *org.apache.flume.clients.log4jappender.Log4jAppender *in MyApp.
>
> But i am getting only body of the message (description). loosing the time,
> thread, Level information.
>
> flume-conf.properties file
> ==================
> agent2Test1.sources = seqGenSrc
> agent2Test1.channels = memoryChannel
> agent2Test1.sinks = loggerSink
>
> # For each one of the sources, the type is defined
> agent2Test1.sources.seqGenSrc.type = avro
> agent2Test1.sources.seqGenSrc.bind=localhost
> agent2Test1.sources.seqGenSrc.port=41414
>
> # interceptors for host and date
> agent2Test1.sources.seqGenSrc.interceptors = time hostInterceptor
> agent2Test1.sources.seqGenSrc.interceptors.hostInterceptor.type =
> org.apache.flume.interceptor.HostInterceptor$Builder
> agent2Test1.sources.seqGenSrc.interceptors.hostInterceptor.hostHeader =
> host
> agent2Test1.sources.seqGenSrc.interceptors.hostInterceptor.useIP = false
> agent2Test1.sources.seqGenSrc.interceptors.hostInterceptor.hostHeader.preserveExisting
> = false
> agent2Test1.sources.seqGenSrc.interceptors.time.type =
> org.apache.flume.interceptor.TimestampInterceptor$Builder
>
> # The channel can be defined as follows.
> agent2Test1.sources.seqGenSrc.channels = memoryChannel
>
> # Each sink's type must be defined
> agent2Test1.sinks.loggerSink.type = hdfs
> agent2Test1.sinks.loggerSink.hdfs.path =
> hdfs://hadoopHost:8020/data/%Y/%m/%d/%{host}/Logs
>
> agent2Test1.sinks.loggerSink.hdfs.fileType = DataStream
>
> #Specify the channel the sink should use
> agent2Test1.sinks.loggerSink.channel = memoryChannel
>
> # Each channel's type is defined.
> agent2Test1.channels.memoryChannel.type = memory
>
> # Other config values specific to each type of channel(sink or source)
> # can be defined as well
> # In this case, it specifies the capacity of the memory channel
> agent2Test1.channels.memoryChannel.capacity = 1000
>
>
>
> Sample java program to generate the log message:
> =====================================
>
>
> package com.test;
>
>
> import org.apache.flume.clients.log4jappender.Log4jAppender;
> import org.apache.log4j.Logger;
> import org.apache.log4j.MDC;
> import org.apache.log4j.PatternLayout;
>
> import java.util.UUID;
>
>
> public class Main {
>     static Logger log = Logger.getLogger(Main.class);
>
>     public static void main(String[] args) {
>         try {
>          Log4jAppender appender = new Log4jAppender();
>             appender.setHostname("localhost");
>             appender.setPort(41414);
>             appender.setLayout(new PatternLayout("%d [%c] (%t) <%X{user}
> %X{field}> %m"));
>          //   appender.setReconnectAttempts(100);
>
>            appender.activateOptions();
>
>             log.addAppender(appender);
>
>             MDC.put("user", "chris");
>           //  while (true) {
>                 MDC.put("field", UUID.randomUUID().toString());
>                 log.info("=====> Hello World");
>                 try {
>                     throw new Exception("Testing");
>                 } catch (Exception e) {
>                     log.error("Gone wrong ===>", e);
>                 }
>             //}
>                 System.in.read();
>                 System.in.read();
>         }
>         catch (Exception e) {
>             e.printStackTrace();
>         }
>     }
> }
>
>
>
> I am missing any config here ?
>
>
>
> --
> Thanks,
> Khadar
>
>


-- 
Thanks,
Khadar

Mime
View raw message