flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tyler Akidau <taki...@google.com.INVALID>
Subject Re: Towards a spec for robust streaming SQL, Part 2
Date Wed, 02 Aug 2017 06:17:44 GMT
Thank you all for the comments/input, I appreciate the time you've put into
this. I've responded to a handful of the major ones. There are some more
I'd like to respond to, but I'm out of time for tonight, so more tomorrow.

-Tyler

On Tue, Aug 1, 2017 at 12:24 PM Julian Hyde <jhyde@apache.org> wrote:

> I have problems with a couple of the axioms: that a SQL object is
> either a table or a stream, but not both; and that a query is bounded
> if and only if it contains no unbounded streams.
>
> I don't have problems with other axioms, such that a query is either
> bounded or unbounded. And I haven't looked in detail at triggering
> semantics; I don't think there will be major issues, but let's clear
> up the 2 problems above first.
>
> I have added a section "Julian’s thoughts on the fundamentals" to the
> end of the document.
>
> Julian
>
>
> On Tue, Aug 1, 2017 at 6:40 AM, Fabian Hueske <fhueske@gmail.com> wrote:
> > As promised, I went of the document and made some comments.
> > I also added a bit of information about the current SQL support in Flink
> > and its internals.
> >
> > Thanks, Fabian
> >
> > 2017-07-30 13:22 GMT+02:00 Shaoxuan Wang <wshaoxuan@gmail.com>:
> >
> >> Hi Tyler,
> >> Thanks for putting all the efforts into a doc. It is really well written
> >> and organized.
> >> I like the most part. The major concern I have is about the "explicit
> >> trigger". I left a few comments towards this and would like to know what
> >> the others think about it.
> >>
> >> Regards,
> >> Shaoxuan
> >>
> >> On Sun, Jul 30, 2017 at 4:43 PM, Fabian Hueske <fhueske@gmail.com>
> wrote:
> >>
> >> > Thanks for the great write up!
> >> >
> >> > I think this s very good starting point for a detailed discussion
> about
> >> > features, syntax and semantics of streaming SQL.
> >> > I'll comment on the document in the next days and describe Flink's
> >> current
> >> > status, our approaches (or planned approaches) and ask a couple of
> >> > questions.
> >> >
> >> > Thanks, Fabian
> >> >
> >> > 2017-07-28 3:05 GMT+02:00 Julian Hyde <jhyde@apache.org>:
> >> >
> >> > > Tyler,
> >> > >
> >> > > Thanks for this. I am reading the document thoroughly and will give
> my
> >> > > feedback in a day or two.
> >> > >
> >> > > Julian
> >> > >
> >> > > > On Jul 25, 2017, at 12:54 PM, Pramod Immaneni <
> >> pramod@datatorrent.com>
> >> > > wrote:
> >> > > >
> >> > > > Thanks for the invitation Tyler. I am sure folks who worked on
the
> >> > > calcite
> >> > > > integration and others would be interested.
> >> > > >
> >> > > > On Tue, Jul 25, 2017 at 12:12 PM, Tyler Akidau
> >> > > <takidau@google.com.invalid>
> >> > > > wrote:
> >> > > >
> >> > > >> +dev@apex.apache.org, since I'm told Apex has a Calcite
> integration
> >> > as
> >> > > >> well. If anyone on the Apex side wants to join in on the
fun,
> your
> >> > input
> >> > > >> would be welcomed!
> >> > > >>
> >> > > >> -Tyler
> >> > > >>
> >> > > >>
> >> > > >> On Mon, Jul 24, 2017 at 4:34 PM Tyler Akidau <takidau@apache.org
> >
> >> > > wrote:
> >> > > >>
> >> > > >>> Hello Flink, Calcite, and Beam dev lists!
> >> > > >>>
> >> > > >>> Linked below is the second document I promised way back
in April
> >> > > >> regarding
> >> > > >>> a collaborative spec for streaming SQL in Beam/Calcite/Flink
(&
> >> > > apologies
> >> > > >>> for the delay; I thought I was nearly done a while back
and then
> >> > > temporal
> >> > > >>> joins expanded to something much larger than expected).
> >> > > >>>
> >> > > >>> To repeat what it says in the doc, my hope is that it
can serve
> >> > various
> >> > > >>> purposes over it's lifetime:
> >> > > >>>
> >> > > >>>   -
> >> > > >>>   - A discussion ground for ironing out any remaining
features
> >> > > necessary
> >> > > >>>   for supporting robust streaming semantics in Calcite
SQL.
> >> > > >>>
> >> > > >>>   - A rough, high-level source of truth for tracking
efforts
> >> underway
> >> > > in
> >> > > >>>   support of this, currently spanning the Calcite, Flink,
and
> Beam
> >> > > >> projects.
> >> > > >>>
> >> > > >>>   - A written specification of the changes that were
made, for
> the
> >> > sake
> >> > > >>>   of understanding the delta after the fact.
> >> > > >>>
> >> > > >>> The first and third points are, IMO, the most important.
AFAIK,
> >> there
> >> > > are
> >> > > >>> a few features missing still that need to be defined
(e.g.,
> >> triggers
> >> > > >>> equivalents via EMIT, robust temporal join support).
I'm also
> >> > > proposing a
> >> > > >>> clear distinction of streams and tables, which I think
is
> >> important,
> >> > > but
> >> > > >>> which I believe is not the approach most folks have been
taking
> in
> >> > this
> >> > > >>> area. Sorting out these open issues and then having a
concise
> >> record
> >> > of
> >> > > >> the
> >> > > >>> solutions adopted will be important for providing a solid
> streaming
> >> > > >>> experience and teaching folks how to use it.
> >> > > >>>
> >> > > >>> At any rate, I would much appreciate it if anyone with
an
> interest
> >> in
> >> > > >> this
> >> > > >>> stuff could please take a look and add comments/suggestions/
> >> > references
> >> > > >> to
> >> > > >>> related work in flight/etc as appropriate. For now please
use
> >> > > >>> comments/suggestions, but if you really want to dive
in with
> edit
> >> > > access,
> >> > > >>> let me know.
> >> > > >>>
> >> > > >>> The doc: http://s.apache.org/streaming-sql-spec
> >> > > >>>
> >> > > >>> -Tyler
> >> > > >>>
> >> > > >>>
> >> > > >>>
> >> > > >>
> >> > >
> >> > >
> >> >
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message