eagle-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Zhou <Daniel.Z...@dataguise.com>
Subject Re: HDFS audit logs with log4j to kafka with kerberos
Date Mon, 20 Feb 2017 01:31:40 GMT
No problem, let me know if you meet other issues.

Regards,
Da

Sent from my iPhone

On Feb 19, 2017, at 1:19 PM, Markovich <amrivkin@gmail.com<mailto:amrivkin@gmail.com>>
wrote:

Hi Da,

Thank you!
Will try to use logstash:)

Regards,
Andrey

2017-02-19 21:35 GMT+03:00 Da Zhou <daveloperzhou@gmail.com<mailto:daveloperzhou@gmail.com>>:
Hi Andrey,

Kafka with kerberos is not supported for now. The workaround is to use Logstash<https://www.elastic.co/guide/en/logstash/current/installing-logstash.html>,
which supports Kerberos SASL (requires plugin version 5.1.0 or later).

To me Logstash is easier to use and debug, so I would suggest you to use logstash :)
Notice that the logstash config file example listed on “http://eagle.apache.org/docs/import-hdfs-auditLog.html”
is for Logstash 1.5, here<https://github.com/apache/eagle/blob/branch-0.4/eagle-assembly/src/main/docs/logstash-kafka-conf.md>
provides another config example for logstash 2.X.

Important:
Some of the kafka -output-plugin settings in config file have changed when it comes to 5.1.X,
so you might need to check their official doc<https://www.elastic.co/guide/en/logstash/current/plugins-outputs-kafka.html>
to update the config file.


Regards,
Da


On Feb 18, 2017, at 9:12 AM, Markovich <amrivkin@gmail.com<mailto:amrivkin@gmail.com>>
wrote:

Hi everyone,

How can I make
log4j.appender.KAFKA_HDFS_AUDIT=org.apache.eagle.log4j.kafka.KafkaLog4jAppender
work with kafka 0.10 with kerberos?

I've configured everything using this note: http://eagle.apache.org/docs/import-hdfs-auditLog.html
(second part) but my kafka broker got nothing.
Also I'm very interested where to look for logs of logj4.appender?

Regards,
Andrey

Mime
View raw message