nifi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joseph Witt (JIRA)" <>
Subject [jira] [Commented] (NIFI-220) Build Processors to interact with Kafka
Date Tue, 06 Jan 2015 00:00:38 GMT


Joseph Witt commented on NIFI-220:

First:  Cool!  It works great.  Easy to setup.  Great docs.  Tests are cool.  Some thoughts/questions:

- PutKafka
-- 'Delivery Guarantee' switches to a numeric value when running.
-- Message Delimiter: I was assuming this would be a delimiter we'd inject in between subsequent
FlowFiles - not that it would be a string used to split input.  The approach as you have it
may offer pretty good throughput though so interesting.  Is that delimeter a byte string?
 Or a UTF-8 string?  I think we'll need to be really clear.  If not a byte sequence then likely
fairly limiting.

- GetKafka
-- Zookeeper Commit Frequency: Can you describe this one a bit more?  It indicates it will
tell ZK every X time units (60 s by default) about what the id of the last message pulled
was.  Should the default be much smaller?  What happens if NiFi is restarted in between? 
Does it keep state locally too?  
-- Message Demarcator:  Is this a UTF-8 string or some other charset?  Or is a byte sequence?

- Combined Names
-- 'Known Brokers' in PutKafka (and is the kafka address).  'ZooKeeper Connection String'
in the case of GetKafka . Is this something where consistency just isn't an option?
-- 'GetKafka' has 'Message Demarcator' and 'PutKafka' has 'message delimeter'.  I think these
are the proper terms but they're transposed.  'GetKafka' adds a delimiter.  PutKafka uses
a demarcator to split.
-- Client Name - What happens if the user chooses to modify these?

These comments are just suggestions.  What do you think?

> Build Processors to interact with Kafka
> ---------------------------------------
>                 Key: NIFI-220
>                 URL:
>             Project: Apache NiFi
>          Issue Type: Task
>          Components: Extensions
>            Reporter: Mark Payne

This message was sent by Atlassian JIRA

View raw message