kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Damian Guy <damian....@gmail.com>
Subject Re: [DISCUSS] KIP-114: KTable materialization and improved semantics
Date Mon, 16 Jan 2017 17:17:26 GMT
Hi Eno,

Thanks for the KIP. Some comments:

   1. I'd probably rename materialized to materialize.
   2. I don't think the addition of the new Log compaction mechanism is
   necessary for this KIP, i.e, the KIP is useful without it. Maybe that
   should be a different KIP?
   3. What will happen when you call materialize on KTable that is already
   materialized? Will it create another StateStore (providing the name is
   different), throw an Exception?
   4. Have you considered overloading the existing KTable operations to add
   a state store name? So if a state store name is provided, then materialize
   a state store? This would be my preferred approach as i don't think
   materialize is always a valid operation.
   5. The materialize method will need ta value Serde as some operations,
   i.e., mapValues, join etc can change the value types
   6. https://issues.apache.org/jira/browse/KAFKA-4609 - might mean that we
   always need to materialize the StateStore for KTable-KTable joins. If that
   is the case, then the KTable Join operators will also need Serde
   information.

Cheers,
Damian


On Mon, 16 Jan 2017 at 16:44 Eno Thereska <eno.thereska@gmail.com> wrote:

> Hello,
>
> We created "KIP-114: KTable materialization and improved semantics" to
> solidify the KTable semantics in Kafka Streams:
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-114%3A+KTable+materialization+and+improved+semantics
> <
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-114:+KTable+materialization+and+improved+semantics
> >
>
> Your feedback is appreciated.
> Thanks
> Eno

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message