calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Using Calcite for Apache Flink's higher-level languages
Date Wed, 09 Dec 2015 15:02:46 GMT
Hi Calcite Folks!

The Apache Flink community is currently looking into how to use Calcite for
optimization of both batch and streaming programs.

We are looking to compile two different kinds of higher level APIs via
Calcite to Flink's APIs:
  - Table API (a LINQ-style DSL)
  - SQL

Our current thought is to use largely the same translation paths for both,
with different entry points into Calcite:
  - the Table API creates a Calcite operator tree directly
  - the SQL interface goes through the full stack, including parser, ...


>From what I have seen so far in Calcite, it looks pretty promising, with
its configurable and extensible rule set, and the pluggable schema/metadata
providers.

A few questions remain for us, to see how feasible this is:

1) Are DAG programs supported? The table API produces operator DAGs, rather
than pure trees. Do DAGs affect/limit the space explored by the optimizer
engine?

2) For streaming programs, we will probably want to add some custom syntax,
specific to Flink's windows. Is it possible to also customize the SQL
dialect of the parser?

These answers are quite crucial for us figure out how to best use Calcite
in our designs. Thanks for helping us...

Greetings,
Stephan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message