Mailing-List: contact dev-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
MIME-Version: 1.0
References: <CAB8MnHVA4KjSj6GeOvKZknxRqgKwC8RLwWnCXcm=DuGW2yrFBA@mail.gmail.com>
 <CAB8MnHU+RQ_YxUT+=nrZYrSFtn15xb68Ps+g4DSxn8vvMpCrjw@mail.gmail.com>
 <CAA7LveEQ4-hV4pW48u+jbQcAmjeF0BX9+wZBXrb6rhPUwWyy7g@mail.gmail.com>
 <F1827533-3D72-4F12-A3E9-8DBE8798B990@apache.org> <CAAdrtT0YRGocfY8tva5o9VGfzYEbN6dmQzx6S+8b8_3681tcig@mail.gmail.com>
 <CAAjCYUVn2_Dc5C+kXz=wEjwDZ8BDVBjXCT-8oz5H81S1Nnv6Ew@mail.gmail.com>
 <CAAdrtT1ej8Y1YFDZ6+sai5y5rO44fE_-ksQ9zvYR5io_Xc7RfA@mail.gmail.com> <CAPSgeES6BSP07azCpPUh=Y-W_VWkL6qOp-e03EWZPx5vi5fWvg@mail.gmail.com>
In-Reply-To: <CAPSgeES6BSP07azCpPUh=Y-W_VWkL6qOp-e03EWZPx5vi5fWvg@mail.gmail.com>
From: Tyler Akidau <takidau@google.com.INVALID>
Date: Wed, 02 Aug 2017 06:17:44 +0000
Message-ID: <CAB8MnHWYKD31PJofNzmoj=f2W6CfTZWLhkP5ahuapFF1X39D4w@mail.gmail.com>
Subject: Re: Towards a spec for robust streaming SQL, Part 2
To: dev@calcite.apache.org
Cc: Dev <dev@flink.apache.org>, dev@beam.apache.org
Content-Type: multipart/alternative; boundary="001a114e56fe7e6b7c0555bf3966"
archived-at: Wed, 02 Aug 2017 06:18:07 -0000

--001a114e56fe7e6b7c0555bf3966
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Thank you all for the comments/input, I appreciate the time you've put into
this. I've responded to a handful of the major ones. There are some more
I'd like to respond to, but I'm out of time for tonight, so more tomorrow.

-Tyler

On Tue, Aug 1, 2017 at 12:24 PM Julian Hyde <jhyde@apache.org> wrote:

> I have problems with a couple of the axioms: that a SQL object is
> either a table or a stream, but not both; and that a query is bounded
> if and only if it contains no unbounded streams.
>
> I don't have problems with other axioms, such that a query is either
> bounded or unbounded. And I haven't looked in detail at triggering
> semantics; I don't think there will be major issues, but let's clear
> up the 2 problems above first.
>
> I have added a section "Julian=E2=80=99s thoughts on the fundamentals" to=
 the
> end of the document.
>
> Julian
>
>
> On Tue, Aug 1, 2017 at 6:40 AM, Fabian Hueske <fhueske@gmail.com> wrote:
> > As promised, I went of the document and made some comments.
> > I also added a bit of information about the current SQL support in Flin=
k
> > and its internals.
> >
> > Thanks, Fabian
> >
> > 2017-07-30 13:22 GMT+02:00 Shaoxuan Wang <wshaoxuan@gmail.com>:
> >
> >> Hi Tyler,
> >> Thanks for putting all the efforts into a doc. It is really well writt=
en
> >> and organized.
> >> I like the most part. The major concern I have is about the "explicit
> >> trigger". I left a few comments towards this and would like to know wh=
at
> >> the others think about it.
> >>
> >> Regards,
> >> Shaoxuan
> >>
> >> On Sun, Jul 30, 2017 at 4:43 PM, Fabian Hueske <fhueske@gmail.com>
> wrote:
> >>
> >> > Thanks for the great write up!
> >> >
> >> > I think this s very good starting point for a detailed discussion
> about
> >> > features, syntax and semantics of streaming SQL.
> >> > I'll comment on the document in the next days and describe Flink's
> >> current
> >> > status, our approaches (or planned approaches) and ask a couple of
> >> > questions.
> >> >
> >> > Thanks, Fabian
> >> >
> >> > 2017-07-28 3:05 GMT+02:00 Julian Hyde <jhyde@apache.org>:
> >> >
> >> > > Tyler,
> >> > >
> >> > > Thanks for this. I am reading the document thoroughly and will giv=
e
> my
> >> > > feedback in a day or two.
> >> > >
> >> > > Julian
> >> > >
> >> > > > On Jul 25, 2017, at 12:54 PM, Pramod Immaneni <
> >> pramod@datatorrent.com>
> >> > > wrote:
> >> > > >
> >> > > > Thanks for the invitation Tyler. I am sure folks who worked on t=
he
> >> > > calcite
> >> > > > integration and others would be interested.
> >> > > >
> >> > > > On Tue, Jul 25, 2017 at 12:12 PM, Tyler Akidau
> >> > > <takidau@google.com.invalid>
> >> > > > wrote:
> >> > > >
> >> > > >> +dev@apex.apache.org, since I'm told Apex has a Calcite
> integration
> >> > as
> >> > > >> well. If anyone on the Apex side wants to join in on the fun,
> your
> >> > input
> >> > > >> would be welcomed!
> >> > > >>
> >> > > >> -Tyler
> >> > > >>
> >> > > >>
> >> > > >> On Mon, Jul 24, 2017 at 4:34 PM Tyler Akidau <takidau@apache.or=
g
> >
> >> > > wrote:
> >> > > >>
> >> > > >>> Hello Flink, Calcite, and Beam dev lists!
> >> > > >>>
> >> > > >>> Linked below is the second document I promised way back in Apr=
il
> >> > > >> regarding
> >> > > >>> a collaborative spec for streaming SQL in Beam/Calcite/Flink (=
&
> >> > > apologies
> >> > > >>> for the delay; I thought I was nearly done a while back and th=
en
> >> > > temporal
> >> > > >>> joins expanded to something much larger than expected).
> >> > > >>>
> >> > > >>> To repeat what it says in the doc, my hope is that it can serv=
e
> >> > various
> >> > > >>> purposes over it's lifetime:
> >> > > >>>
> >> > > >>>   -
> >> > > >>>   - A discussion ground for ironing out any remaining features
> >> > > necessary
> >> > > >>>   for supporting robust streaming semantics in Calcite SQL.
> >> > > >>>
> >> > > >>>   - A rough, high-level source of truth for tracking efforts
> >> underway
> >> > > in
> >> > > >>>   support of this, currently spanning the Calcite, Flink, and
> Beam
> >> > > >> projects.
> >> > > >>>
> >> > > >>>   - A written specification of the changes that were made, for
> the
> >> > sake
> >> > > >>>   of understanding the delta after the fact.
> >> > > >>>
> >> > > >>> The first and third points are, IMO, the most important. AFAIK=
,
> >> there
> >> > > are
> >> > > >>> a few features missing still that need to be defined (e.g.,
> >> triggers
> >> > > >>> equivalents via EMIT, robust temporal join support). I'm also
> >> > > proposing a
> >> > > >>> clear distinction of streams and tables, which I think is
> >> important,
> >> > > but
> >> > > >>> which I believe is not the approach most folks have been takin=
g
> in
> >> > this
> >> > > >>> area. Sorting out these open issues and then having a concise
> >> record
> >> > of
> >> > > >> the
> >> > > >>> solutions adopted will be important for providing a solid
> streaming
> >> > > >>> experience and teaching folks how to use it.
> >> > > >>>
> >> > > >>> At any rate, I would much appreciate it if anyone with an
> interest
> >> in
> >> > > >> this
> >> > > >>> stuff could please take a look and add comments/suggestions/
> >> > references
> >> > > >> to
> >> > > >>> related work in flight/etc as appropriate. For now please use
> >> > > >>> comments/suggestions, but if you really want to dive in with
> edit
> >> > > access,
> >> > > >>> let me know.
> >> > > >>>
> >> > > >>> The doc: http://s.apache.org/streaming-sql-spec
> >> > > >>>
> >> > > >>> -Tyler
> >> > > >>>
> >> > > >>>
> >> > > >>>
> >> > > >>
> >> > >
> >> > >
> >> >
> >>
>

--001a114e56fe7e6b7c0555bf3966--