samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rick Mangi <r...@chartbeat.com>
Subject Re: Best way to log from inside a Samza task?
Date Fri, 26 Jun 2015 18:03:21 GMT
If you do something like this in your log4j.xml

  <root>
        <priority value="INFO" />
        <appender-ref ref="RollingAppender" />
    </root>

    <logger name="cbsamza" additivity="false">
      <level value=“DEBUG" />
      <appender-ref ref="RollingAppender" />
    </logger>

the root controls samza’s logging and the logger controls your own… I haven’t managed
to get the imx configuration working yet.




> On Jun 26, 2015, at 1:50 PM, jason@marketingscience.co wrote:
> 
> I was almost there. Got it now. Thanks for your help Rick. 
> 
> 
> 
> 
> Cheers, 
> 
> 
> 
> 
> Jason
> 
> 
> 
> 
> 
> 
> 
> 
> 
> On Friday, Jun 26, 2558 at 11:43, Rick Mangi <rick@chartbeat.com>, wrote:
> Hey Jason,
> 
> 
> If you configure log4j as described here: http://samza.apache.org/learn/documentation/0.9/jobs/logging.html
<http://samza.apache.org/learn/documentation/0.9/jobs/logging.html>
> 
> 
> Your log statements will wind up in the samza-container logs which you can get to via
the application master gui.
> 
> 
> hth,
> 
> 
> Rick
> 
> 
> 
>> On Jun 26, 2015, at 1:39 PM, jason@marketingscience.co wrote:
> 
>> 
> 
>> Hello,
> 
>> 
> 
>> 
> 
>> I am working on a basic Samza task that pulls from one Kafka topic and writes to
another. This task runs in Yarn but the Output topic does not contain any data. 
> 
>> 
> 
>> 
> 
>> In order to troubleshoot this more effectively I would like to log the incoming message
as my example below. Ideally, I would like to be able to see the log messages in Yarn, maybe
in the .out files in the /logs directory. 
> 
>> 
> 
>> 
> 
>> Any advice is appreciated. 
> 
>> 
> 
>> 
> 
>> 
> 
>> Here is my task:
> 
>> 
> 
>> 
> 
>> package com.project.samza.tasks;
> 
>> 
> 
>> 
> 
>> import java.util.Map;
> 
>> import org.apache.samza.system.IncomingMessageEnvelope;
> 
>> import org.apache.samza.system.OutgoingMessageEnvelope;
> 
>> import org.apache.samza.system.SystemStream;
> 
>> import org.apache.samza.task.MessageCollector;
> 
>> import org.apache.samza.task.StreamTask;
> 
>> import org.apache.samza.task.TaskCoordinator;
> 
>> 
> 
>> 
> 
>> public class exampleStreamTask implements StreamTask {
> 
>>  private static final SystemStream OUTPUT_STREAM = new SystemStream(“kafka”,
“new-topic-test”);
> 
>> 
> 
>> 
> 
>>  @Override
> 
>>  public void process(IncomingMessageEnvelope envelope,
> 
>>      MessageCollector collector,
> 
>>      TaskCoordinator coordinator) {
> 
>>          String msg = (String) envelope.getMessage();
> 
>>          System.out.println(msg);
> 
>>          collector.send(new OutgoingMessageEnvelope(OUTPUT_STREAM, msg));
> 
>>  }
> 
>> }
> 
>> 
> 
>> 
> 
>> Thanks, 
> 
>> 
> 
>> 
> 
>> Jason


Mime
View raw message