kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Randall Hauch <rha...@gmail.com>
Subject Re: [DISCUSS] KIP-154 Add Kafka Connect configuration properties for creating internal topics
Date Sat, 06 May 2017 02:18:34 GMT
Thanks, Gwen. 

Switching to low-priority is a great idea.

The default value for the replication factor configuration is 3, since that makes sense and
is safe for production. Using the default values in the example would mean it could only be
run against a Kafka cluster with a minimum of 3 nodes. I propose overriding the example's
replication factor configurations to be 1 so that the examples could be run on any sized cluster.

The rejected alternatives mentions why the implementation doesn't try to be too smart by calculating
the replication factor.

Best regards, 


> On May 5, 2017, at 8:02 PM, Gwen Shapira <gwen@confluent.io> wrote:
> Looks great to me :)
> Just one note - configurations have levels (which reflect in the docs) - I
> suggest putting the whole thing as LOW. Most users will never need to worry
> about these. For same reason I recommend leaving them out of the example
> config files - we already have issues with users playing with configs
> without understanding what they are doing and not liking the results.
>> On Fri, May 5, 2017 at 3:42 PM, Randall Hauch <rhauch@gmail.com> wrote:
>> Hi, all.
>> I've been working on KAFKA-4667 to change the distributed worker of Kafka
>> Connect to look for the topics used to store connector and task
>> configurations, offsets, and status, and if those tasks do not exist to
>> create them using the new AdminClient. To make this as useful as possible
>> and to minimize the need to still manually create the topics, I propose
>> adding several new distributed worker configurations to specify the
>> partitions and replication factor for these topics, and have outlined them
>> in "KIP-154 Add Kafka Connect configuration properties for creating
>> internal topics".
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>> 154+Add+Kafka+Connect+configuration+properties+for+
>> creating+internal+topics
>> Please take a look and provide feedback. Thanks!
>> Best regards,
>> Randall
> -- 
> *Gwen Shapira*
> Product Manager | Confluent
> 650.450.2760 | @gwenshap
> Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> <http://www.confluent.io/blog>

View raw message