[ https://issues.apache.org/jira/browse/NIFI-220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14265447#comment-14265447
]
Mark Payne commented on NIFI-220:
---------------------------------
Joe,
PutKafka:
- Delivery Guarantee does switch to a numeric value when running, as Kafka wants them as
-1, 0, or 1. But I didn't want to expose those arbitrary values in the UI, so I provided names.
- Message Delimiter is a UTF-8 String. I considered requiring a hex-encoded byte string,
but that's pretty terrible if the data is text. Could make it configurable but then it's a
bit of a weird property to add. Could check if it starts with '0x' and if so expect a hex-encoded
byte sequence?
GetKafka:
- We do not store these values locally. If NiFi is restarted, it could result in duplication
of 'up to X amount of time worth of data'. Kafka's default is 60 seconds, so I chose to use
their default.
- Message Demarcator: Same as Message Delimiter in PutKafka.
Combined:
- The Known Brokers vs. ZooKeeper Connection String, unfortunately, is what Kafka exposes.
Can't make them consistent with their API.
- GetKafka is configured with a Demarcator, as it is demarcating the messages that it receives.
PutKafka has a delimiter, as the incoming data is delimited using this delimiter. I can see
your viewpoint, too, though... honestly, I think either way works.
- Client Name is mapped to 'client.id' property in Kafka config. Their docs state "The client
id is a user-specified string sent in each request to help trace calls. It should logically
identify the application making the request." So modifying that should be fine.
I think the biggest thing that we want to consider here is the delimiter/demarcator: What
do you think of using a prefix of "0x" to indicate hex byte sequence, otherwise using a UTF-8
Encoded string?
> Build Processors to interact with Kafka
> ---------------------------------------
>
> Key: NIFI-220
> URL: https://issues.apache.org/jira/browse/NIFI-220
> Project: Apache NiFi
> Issue Type: Task
> Components: Extensions
> Reporter: Mark Payne
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
|