samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yi Pan <nickpa...@gmail.com>
Subject Re: [VOTE] SEP-8: Add in-memory system consumer & producer
Date Wed, 04 Oct 2017 21:41:28 GMT
+1. The updated design looks good to me! Looking forward to its
implementation.

Minor: if this SEP has direct dependency on SEP-2 and other
ApplicationRunner related API changes, please make it clear (i.e. what we
can implement w/o being blocked and what need to wait till the API changes
are committed).

-Yi

On Tue, Sep 26, 2017 at 10:58 AM, Jagadish Venkatraman <
jagadish1989@gmail.com> wrote:

> LGTM , +1 on the overall design. This will drastically improve testing of
> Samza applications!
>
> --
> Jagdish
>
> On Thu, Sep 14, 2017 at 1:23 AM, Yi Pan <nickpan47@gmail.com> wrote:
>
> > Hi, Bharath,
> >
> > Overall looks good! I have the following comments:
> >
> > i) Question on the Type of IME + data partition:
> >
> > How do we enforce that user adds IME w/ the expected partition id to the
> > corresponding sub-collection?
> >
> >
> >
> > ii) In the architecture graph, what's the difference between SSP queues
> and
> > Data source/sink? What is the layer exposed to the user (I.e.
> programmer)?
> >
> >
> >
> > ii) Agree w/ the approach to use a customized queues managed by the
> admin.
> > However, the reason not to use BEM is not very clear. For the matter of
> > fact, BEM is just one optional base class for SystemConsumer
> > implementation.
> > Not sure why we necessarily need to be limited by BEM.
> >
> >
> >
> > iii) In the code examples,
> >
> > A) what's the difference between durable state vs non-durable state in
> > highlevel API? I don't see any difference. Also, the SEP has clearly
> > described that the design is only for InMemory input/output/intermediate
> > streams. I noticed that you added changelog as inputs in low-level API.
> But
> > it is not clear how this changelog is defined and why it is an input to
> the
> > application???
> >
> > B) the code example for checkpoint is empty and we have stated that we
> > won't support checkpoint in this SEP. Can we remove it?
> >
> >
> > Thanks!
> >
> >
> > -Yi
> >
> > On Wed, Sep 6, 2017 at 2:06 PM, xinyu liu <xinyuliu.us@gmail.com> wrote:
> >
> > > +1 on the overall design. This will make testing a lot easier!
> > >
> > > Thanks,
> > > Xinyu
> > >
> > > On Wed, Sep 6, 2017 at 10:45 AM, Bharath Kumara Subramanian <
> > > codin.martial@gmail.com> wrote:
> > >
> > > > Hi all,
> > > >
> > > > Can you please vote for SEP-8?
> > > > You can find the design document here
> > > > <https://cwiki.apache.org/confluence/pages/viewpage.
> > > action?pageId=71013043
> > > > >.
> > > >
> > > > Thanks,
> > > > Bharath
> > > >
> > >
> >
>
>
>
> --
> Jagadish V,
> Graduate Student,
> Department of Computer Science,
> Stanford University
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message