flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: SQL on Flink
Date Wed, 27 May 2015 13:49:43 GMT
I didn't know that paper...  Thanks for sharing.

I've worked on a SQL layer for Stratosphere some time ago, using Apache
Calcite (called Optiq back then). I think the project provides a lot of
very good tooling for creating a SQL layer. So if we decide to go for SQL
on Flink, I would suggest to use Calcite.
I can also help you a bit with Calcite to get started with it.

I agree with Fabian that it would probably make more sense for now to
enhance the Table API.
I think the biggest limitation right now is that it only supports POJOs.
We should also support Tuples (I know thats difficult to do), data from
HCatalog (that includes parquet & orc), JSON, ...
Then, I would add filter and projection pushdown into the table API.



On Tue, May 26, 2015 at 10:03 PM, Ted Dunning <ted.dunning@gmail.com> wrote:

> It would also be relatively simple (I think) to retarget drill to Flink if
> Flink doesn't provide enough typing meta-data to do traditional SQL.
>
>
>
> On Tue, May 26, 2015 at 12:52 PM, Fabian Hueske <fhueske@gmail.com> wrote:
>
> > Hi,
> >
> > Flink's Table API is pretty close to what SQL provides. IMO, the best
> > approach would be to leverage that and build a SQL parser (maybe together
> > with a logical optimizer) on top of the Table API. Parser (and optimizer)
> > could be built using Apache Calcite which is providing exactly this.
> >
> > Since the Table API is still a fairly new component and not very feature
> > rich, it might make sense to extend and strengthen it before putting
> > something major on top.
> >
> > Cheers, Fabian
> >
> > 2015-05-26 21:38 GMT+02:00 Timo Walther <twalthr@apache.org>:
> >
> > > Hey everyone,
> > >
> > > I would be interested in having a complete SQL API in Flink. How is the
> > > status there? Is someone already working on it? If not, I would like to
> > > work on it. I found http://ijcsi.org/papers/IJCSI-12-1-1-169-174.pdf
> but
> > > I couldn't find anything on the mailing list or Jira. Otherwise I would
> > > open an issue and start a discussion about it there.
> > >
> > > Regards,
> > > Timo
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message