kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax" <matth...@confluent.io>
Subject Re: [DISCUSS] KIP-253: Support in-order message delivery with partition expansion
Date Fri, 23 Feb 2018 02:24:22 GMT
Dong,

thanks a lot for the KIP!

Can you elaborate how this would work for compacted topics? If it does
not work for compacted topics, I think Streams API cannot allow to scale
input topics.

This question seems to be particularly interesting for deleting
partitions: assume that a key is never (or for a very long time)
updated, a partition cannot be deleted.


-Matthias


On 2/22/18 5:19 PM, Jay Kreps wrote:
> Hey Dong,
> 
> Two questions:
> 1. How will this work with Streams and Connect?
> 2. How does this compare to a solution where we physically split partitions
> using a linear hashing approach (the partition number is equivalent to the
> hash bucket in a hash table)? https://en.wikipedia.org/wiki/Linear_hashing
> 
> -Jay
> 
> On Sat, Feb 10, 2018 at 3:35 PM, Dong Lin <lindong28@gmail.com> wrote:
> 
>> Hi all,
>>
>> I have created KIP-253: Support in-order message delivery with partition
>> expansion. See
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>> 253%3A+Support+in-order+message+delivery+with+partition+expansion
>> .
>>
>> This KIP provides a way to allow messages of the same key from the same
>> producer to be consumed in the same order they are produced even if we
>> expand partition of the topic.
>>
>> Thanks,
>> Dong
>>
> 


Mime
View raw message