eagle-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Da Zhou <daveloperz...@gmail.com>
Subject Re: HDFS audit logs with log4j to kafka with kerberos
Date Sun, 19 Feb 2017 18:35:44 GMT
Hi Andrey,

Kafka with kerberos is not supported for now. The workaround is to use Logstash <https://www.elastic.co/guide/en/logstash/current/installing-logstash.html>,
which supports Kerberos SASL (requires plugin version 5.1.0 or later).

To me Logstash is easier to use and debug, so I would suggest you to use logstash :)
Notice that the logstash config file example listed on “http://eagle.apache.org/docs/import-hdfs-auditLog.html
<http://eagle.apache.org/docs/import-hdfs-auditLog.html>” is for Logstash 1.5, here
<https://github.com/apache/eagle/blob/branch-0.4/eagle-assembly/src/main/docs/logstash-kafka-conf.md>
provides another config example for logstash 2.X.  

Important:
Some of the kafka -output-plugin settings in config file have changed when it comes to 5.1.X,
so you might need to check their official doc <https://www.elastic.co/guide/en/logstash/current/plugins-outputs-kafka.html>
to update the config file.


Regards,
Da


On Feb 18, 2017, at 9:12 AM, Markovich <amrivkin@gmail.com <mailto:amrivkin@gmail.com>>
wrote:

> Hi everyone, 
> 
> How can I make 
> log4j.appender.KAFKA_HDFS_AUDIT=org.apache.eagle.log4j.kafka.KafkaLog4jAppender
> work with kafka 0.10 with kerberos?
> 
> I've configured everything using this note: http://eagle.apache.org/docs/import-hdfs-auditLog.html
<http://eagle.apache.org/docs/import-hdfs-auditLog.html> (second part) but my kafka
broker got nothing. 
> Also I'm very interested where to look for logs of logj4.appender?
> 
> Regards,
> Andrey

Mime
View raw message