apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Weise <...@apache.org>
Subject Re: [EXTERNAL] kafka
Date Tue, 01 Nov 2016 16:54:00 GMT
IMO this should be added here:

http://apex.apache.org/docs/malhar/operators/kafkaInputOperator/


On Tue, Nov 1, 2016 at 5:24 PM, hsy541@gmail.com <hsy541@gmail.com> wrote:

> Hey Raja,
>
> The setup for secure kafka input operator is not easy. You can follow
> these steps.
> 1. Follow kafka document to setup your brokers properly (
> https://kafka.apache.org/090/documentation.html#security_overview)
> 2. You have to manually create the client JAAS file (
> https://kafka.apache.org/090/documentation.html#security_overview)
>    a sample file would look like this:
>      KafkaClient {
>
>         com.sun.security.auth.module.Krb5LoginModule required
>         useKeyTab=true
>         storeKey=true
>         keyTab="/etc/security/keytabs/kafka_client.keytab"
>         principal="kafka-client-1@EXAMPLE.COM";
>     };
>
> 3. On the operator you have to set the attribute JVM_OPTS
>  -Djava.security.auth.login.config=/etc/kafka/kafka_client_jaas.conf
>
> 4. On the operator you have to set the consumerProperties, for example
>
> set dt.operator.$your_operator_name.{consumerProps.security.protocol}  to
>  SASL or other security function you use
> set dt.operator.$your_operator_name.{consumerPrope.sasl.
> kerberos.service.name} to kafka
>
>
> Hope this would help you!
>
>
> Regards,
> Siyuan
>
>
>
> On Sun, Oct 30, 2016 at 10:56 PM, Raja.Aravapalli <
> Raja.Aravapalli@target.com> wrote:
>
>>
>>
>> Hi Team,
>>
>>
>>
>> Can someone pls help me with below requested information ?
>>
>>
>>
>> Does apache apex have any inbuilt kafka input operator to read from Kafka
>> 0.9 secure kafka topics?
>>
>>
>>
>> Thanks a lot.
>>
>>
>>
>> Regards,
>>
>> Raja.
>>
>>
>>
>> *From: *"Raja.Aravapalli" <Raja.Aravapalli@target.com>
>> *Reply-To: *"users@apex.apache.org" <users@apex.apache.org>
>> *Date: *Monday, October 24, 2016 at 2:29 PM
>> *To: *"users@apex.apache.org" <users@apex.apache.org>
>> *Subject: *[EXTERNAL] kafka
>>
>>
>>
>>
>>
>> Hi,
>>
>>
>>
>> Do we have any kaka operator readily available to consume messages from
>> secure kafka topics in kafka 0.9 clusters?
>>
>>
>>
>> Thanks a lot.
>>
>>
>>
>>
>>
>> Regards,
>>
>> Raja.
>>
>
>

Mime
View raw message