kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Panuwat Anawatmongkhon <panuwat.anawatmongk...@gmail.com>
Subject Re: [DISCUSS] KIP-233: Simplify StreamsBuilder#addGlobalStore
Date Mon, 19 Mar 2018 08:47:31 GMT
Hi Matthias,
I am not sure weather I understand the impact correctly. With the old api
user have to specify name for state store and the name is used for auto
generated topics for fault tolerance, so when replace the old api usage
with the new one we don’t have any kind of auto renaming so the backup
topic name will be inconsistent.
Cheers,
Benz

On Mon, 19 Mar 2561 at 10:59 Panuwat Anawatmongkhon <
panuwat.anawatmongkhon@gmail.com> wrote:

> I couldn’t find this thread in mailing list. this reply is just to trigger
> it up so I can include it in KIP
>
> On Thu, 28 Dec 2560 at 03:07 Matthias J. Sax <matthias@confluent.io>
> wrote:
>
>> @Matthias: just wanted to follow up on your question:
>>
>> >>>> I wanted to double check. If I understand the proposal, it would
>> replace
>> >>>> the explicit name with a name that is dynamically generated using
the
>> >>>> AtomicInteger index. Would this affect the naming of any internally
>> >>>> generated topics?
>>
>> I think it would -- note, that the old API will not be removed but
>> deprecated -- thus, you can still update without any issues staying with
>> the old API -- only if you start to use the new API, it could impact an
>> application.
>>
>> It should still be possible to upgrade to the new API if you invest the
>> time to rename the corresponding topics correctly -- this will only work
>> if you use a new application id or take your application offline though.
>>
>> -Matthias
>>
>>
>> On 12/16/17 10:17 AM, Panuwat Anawatmongkhon wrote:
>> > Hi all,
>> > I would like to start the vote thread tomorrow, feel free to ask if
>> there
>> > is any concern.
>> > Thank you
>> >
>> > On Thu, 7 Dec 2560 at 19:22 Panuwat Anawatmongkhon <
>> > panuwat.anawatmongkhon@gmail.com> wrote:
>> >
>> >>
>> >> Yes, Matthias.
>> >> The object will be used togerther with function table and function
>> stream.
>> >> I didn’t see how this will affect other part but if you do, please
>> explain
>> >> more on how this will affect generated topic name.
>> >> Thank you
>> >> Panuwat
>> >>
>> >>
>> >> On Thu, 7 Dec 2560 at 00:01 Matthias Margush <
>> matthias.margush@gmail.com>
>> >> wrote:
>> >>
>> >>> Hi.
>> >>>
>> >>> I wanted to double check. If I understand the proposal, it would
>> replace
>> >>> the explicit name with a name that is dynamically generated using the
>> >>> AtomicInteger index. Would this affect the naming of any internally
>> >>> generated topics?
>> >>>
>> >>> On Wed, Dec 6, 2017 at 7:59 AM Panuwat Anawatmongkhon <
>> >>> panuwat.anawatmongkhon@gmail.com> wrote:
>> >>>
>> >>>> Thanks Bill.
>> >>>>
>> >>>> I can't think of reason to keep the old method too so if there is
no
>> >>>> further discussion by tomorrow, I would like to start the vote
>> thread.
>> >>>>
>> >>>> On Tue, Dec 5, 2017 at 10:38 PM, Bill Bejeck <bbejeck@gmail.com>
>> wrote:
>> >>>>
>> >>>>> Hi Panuwat,
>> >>>>>
>> >>>>> Thanks for the KIP, overall looks good to me.
>> >>>>>
>> >>>>> I want to play the devil's advocate for a second and ask do
we want
>> to
>> >>>> keep
>> >>>>> the older method with the extra parameters vs. deprecation?
>> >>>>>
>> >>>>> Although ATM I can't think of a good reason to keep the old
method
>> >>> with
>> >>>> the
>> >>>>> extra parameters.
>> >>>>>
>> >>>>> Thanks,
>> >>>>> Bill
>> >>>>>
>> >>>>> On Tue, Dec 5, 2017 at 5:48 AM, Ted Yu <yuzhihong@gmail.com>
wrote:
>> >>>>>
>> >>>>>> Fine by me.
>> >>>>>>
>> >>>>>> On Tue, Dec 5, 2017 at 2:45 AM, Panuwat Anawatmongkhon <
>> >>>>>> panuwat.anawatmongkhon@gmail.com> wrote:
>> >>>>>>
>> >>>>>>> Thank you, Matthias.
>> >>>>>>>
>> >>>>>>> Ted,
>> >>>>>>> How about this.
>> >>>>>>>
>> >>>>>>> String globalTopicName = "testGlobalTopic";
>> >>>>>>> String globalStoreName = "testAddGlobalStore";
>> >>>>>>> final StreamsBuilder builder = new StreamsBuilder();
>> >>>>>>> final KeyValueStoreBuilder globalStoreBuilder =
>> >>>>>>> EasyMock.createNiceMock(KeyValueStoreBuilder.class);
>> >>>>>>>
>> >>>>
>> EasyMock.expect(globalStoreBuilder.name()).andReturn(globalStoreName).
>> >>>>>>> anyTimes();
>> >>>>>>> EasyMock.replay(globalStoreBuilder);
>> >>>>>>> builder.addGlobalStore(globalStoreBuilder,globalTopicName,new
>> >>>>>>> ConsumedInternal(),new MockProcessorSupplier());
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Tue, Dec 5, 2017 at 4:58 AM, Matthias J. Sax <
>> >>>> matthias@confluent.io
>> >>>>>>
>> >>>>>>> wrote:
>> >>>>>>>
>> >>>>>>>> Panuwat,
>> >>>>>>>>
>> >>>>>>>> Thanks a lot for the KIP!
>> >>>>>>>>
>> >>>>>>>> Just one nit: `does not follow provide a good` ->
spelling:
>> >>> remove
>> >>>>>>>> `follow` ?
>> >>>>>>>>
>> >>>>>>>> Otherwise, looks good to me.
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> -Matthias
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On 12/4/17 10:49 AM, Ted Yu wrote:
>> >>>>>>>>> Looks like you're implying logic similar to
this:
>> >>>>>>>>>
>> >>>>>>>>>     public synchronized <K, V> GlobalKTable<K,
V>
>> >>>> globalTable(final
>> >>>>>>>> String
>> >>>>>>>>> topic,
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>  final
>> >>>>>>>>> Consumed<K, V> consumed) {
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> StreamsBuilder is returned instead of GlobalKTable.
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> Can you add code snippet showing how the new
API is used ?
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Mon, Dec 4, 2017 at 10:09 AM, Panuwat Anawatmongkhon
<
>> >>>>>>>>> panuwat.anawatmongkhon@gmail.com> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> What i am thinking right now is using the
same approach as
>> >>>>>>>>>> org.apache.kafka.streams.kstream.internals.
>> >>>>> InternalStreamsBuilder#
>> >>>>>>>>>> globalTable
>> >>>>>>>>>>
>> >>>>>>>>>> On Mon, 4 Dec 2560 at 23:10 Ted Yu <yuzhihong@gmail.com>
>> >>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> Can you describe how sourceName is inferred
based on the new
>> >>>> API
>> >>>>> ?
>> >>>>>>>>>>>
>> >>>>>>>>>>> Please fill out JIRA number.
>> >>>>>>>>>>>
>> >>>>>>>>>>> BTW here is the URL for the KIP:
>> >>>>>>>>>>>
>> >>>>>>>>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
>> >>>>>>> 233%3A+Simplify+
>> >>>>>>>>>> StreamsBuilder%23addGlobalStore
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Mon, Dec 4, 2017 at 7:39 AM, Panuwat
Anawatmongkhon <
>> >>>>>>>>>>> panuwat.anawatmongkhon@gmail.com>
wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> Hi all,
>> >>>>>>>>>>>> I created a KIP.
>> >>>>>>>>>>>> https://cwiki.apache.org/confluence/display/KAFKA/
>> >>>>>>> KIP233%3A+Simplify+
>> >>>>>>>>>>>> StreamsBuilder%23addGlobalStore
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Cheers,
>> >>>>>>>>>>>> Benz
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>>>>
>> >>>>
>> >>>
>> >>
>> >
>>
>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message