flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@gmail.com>
Subject Re: [DISCUSS] Table API / SQL features for Flink 1.4.0
Date Fri, 16 Jun 2017 08:07:05 GMT
Thanks for your response Shaoxuan,

My "Table-table join with retraction" is probably the same as your
"unbounded stream-stream join with retraction".
Basically, a join between two dynamic tables with unique keys (either
because of an upsert stream->table conversion or an unbounded aggregation).

Best, Fabian

2017-06-16 0:56 GMT+02:00 Shaoxuan Wang <wshaoxuan@gmail.com>:

> Nice timing, Fabian!
>
> Your checklist aligns our plans very well. Here are the things we are
> working on & planning to contribute to release 1.4:
> 1. DDL (with property waterMark config for source-table, and emit config on
> result-table)
> 2. unbounded stream-stream joins (with retraction supported)
> 3. backend state user interface for UDAGG
> 4. UDOP (as oppose to UDF(scalars to scalar)/UDTF(scalar to
> table)/UDAGG(table to scalar), this allows user to define a table to table
> conversion business logic)
>
> Some of them already have PR/jira, while some are not. We will send out the
> design doc for the missing ones very soon. Looking forward to the 1.4
> release.
>
> Btw, what is "Table-Table (with retraction)" you have mentioned in your
> plan?
>
> Regards,
> Shaoxuan
>
>
>
> On Thu, Jun 15, 2017 at 10:29 PM, Fabian Hueske <fhueske@gmail.com> wrote:
>
> > Hi everybody,
> >
> > I would like to start a discussion about the targeted feature set of the
> > Table API / SQL for Flink 1.4.0.
> > Flink 1.3.0 was released about 2 weeks ago and we have 2.5 months (~11
> > weeks, until begin of September) left until the feature freeze for Flink
> > 1.4.0.
> >
> > I think it makes sense to start with a collection of desired features.
> Once
> > we have a list of requested features, we might want to prioritize and
> maybe
> > also assign responsibilities.
> >
> > When we prioritize, we should keep in mind that:
> > - we want to have a consistent API. Larger features should be developed
> in
> > a feature branch first.
> > - the next months are typical time for vacations
> > - we have been bottlenecked by committer resources in the last release.
> >
> > I think the following features would be a nice addition to the current
> > state:
> >
> > - Conversion of a stream into an upsert table (with retraction, updating
> to
> > the last row per key)
> > - Joins for streaming tables
> >   - Stream-Stream (time-range predicate) there is already a PR for
> > processing time joins
> >   - Table-Table (with retraction)
> > - Support for late arriving records in group window aggregations
> > - Exposing a keyed result table as queryable state
> >
> > Which features are others looking for?
> >
> > Cheers,
> > Fabian
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message