kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Kreps <...@confluent.io>
Subject Re: [DISCUSS] KIP-120: Cleanup Kafka Streams builder API
Date Tue, 14 Mar 2017 04:42:06 GMT
Hey Matthias,

Make sense, I'm more advocating for removing the word topology than any
particular new replacement.

-Jay

On Mon, Mar 13, 2017 at 12:30 PM, Matthias J. Sax <matthias@confluent.io>
wrote:

> Jay,
>
> thanks for your feedback
>
> > What if instead we called it KStreamsBuilder?
>
> That's the current name and I personally think it's not the best one.
> The main reason why I don't like KStreamsBuilder is, that we have the
> concepts of KStreams and KTables, and the builder creates both. However,
> the name puts he focus on KStream and devalues KTable.
>
> I understand your argument, and I am personally open the remove the
> "Topology" part, and name it "StreamsBuilder". Not sure what others
> think about this.
>
>
> About Processor API: I like the idea in general, but I thinks it's out
> of scope for this KIP. KIP-120 has the focus on removing leaking
> internal APIs and do some cleanup how our API reflects some concepts.
>
> However, I added your idea to API discussion Wiki page and we take if
> from there:
> https://cwiki.apache.org/confluence/display/KAFKA/
> Kafka+Streams+Discussions
>
>
>
> -Matthias
>
>
> On 3/13/17 11:52 AM, Jay Kreps wrote:
> > Two things:
> >
> >    1. This is a minor thing but the proposed new name for KStreamBuilder
> >    is StreamsTopologyBuilder. I actually think we should not put
> topology in
> >    the name as topology is not a concept you need to understand at the
> >    kstreams layer right now. I'd think of three categories of concepts:
> (1)
> >    concepts you need to understand to get going even for a simple
> example, (2)
> >    concepts you need to understand to operate and debug a real
> production app,
> >    (3) concepts we truly abstract and you don't need to ever understand.
> I
> >    think in the kstream layer topologies are currently category (2), and
> this
> >    is where they belong. By introducing the name in even the simplest
> example
> >    it means the user has to go read about toplogies to really understand
> even
> >    this simple snippet. What if instead we called it KStreamsBuilder?
> >    2. For the processor api, I think this api is mostly not for end
> users.
> >    However this are a couple cases where it might make sense to expose
> it. I
> >    think users coming from Samza, or JMS's MessageListener (
> >    https://docs.oracle.com/javaee/7/api/javax/jms/MessageListener.html)
> >    understand a simple callback interface for message processing. In
> fact,
> >    people often ask why Kafka's consumer doesn't provide such an
> interface.
> >    I'd argue we do, it's KafkaStreams. The only issue is that the
> processor
> >    API documentation is a bit scary for a person implementing this type
> of
> >    api. My observation is that people using this style of API don't do a
> lot
> >    of cross-message operations, then just do single message operations
> and use
> >    a database for anything that spans messages. They also don't factor
> their
> >    code into many MessageListeners and compose them, they just have one
> >    listener that has the complete handling logic. Say I am a user who
> wants to
> >    implement a single Processor in this style. Do we have an easy way to
> do
> >    that today (either with the .transform/.process methods in kstreams
> or with
> >    the topology apis)? Is there anything we can do in the way of trivial
> >    helper code to make this better? Also, how can we explain that
> pattern to
> >    people? I think currently we have pretty in-depth docs on our apis
> but I
> >    suspect a person trying to figure out how to implement a simple
> callback
> >    might get a bit lost trying to figure out how to wire it up. A simple
> five
> >    line example in the docs would probably help a lot. Not sure if this
> is
> >    best addressed in this KIP or is a side comment.
> >
> > Cheers,
> >
> > -Jay
> >
> > On Fri, Feb 3, 2017 at 3:33 PM, Matthias J. Sax <matthias@confluent.io>
> > wrote:
> >
> >> Hi All,
> >>
> >> I did prepare a KIP to do some cleanup some of Kafka's Streaming API.
> >>
> >> Please have a look here:
> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> >> 120%3A+Cleanup+Kafka+Streams+builder+API
> >>
> >> Looking forward to your feedback!
> >>
> >>
> >> -Matthias
> >>
> >>
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message