beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Julian Hyde (JIRA)" <>
Subject [jira] [Commented] (BEAM-301) Add a Beam SQL DSL
Date Fri, 14 Apr 2017 00:52:41 GMT


Julian Hyde commented on BEAM-301:

[~takidau], +1 to that approach.

In the short term I think we will have multiple variants of streaming SQL. Calcite will support
STREAM and using the same identifier for streams/tables, and will provide a [switch|]
so that Beam can disable them. Over the longer term I will try make the case that these features
are useful (or I will fail, and these features will wither away). I can't really make the
case until we have features like self-join of a stream to its own history.

The crucial query which illustrates this is this one:

{code}select stream *
from Orders as o
where units > (
  select avg(units)
  from Orders as h
  where h.productId = o.productId
  and h.rowtime > o.rowtime - interval ‘1’ year){code}

It combines the {{Orders}} stream with its own history. But after the query has been running
for a while, the records that passed through the stream will have entered the history. The
history relation {{h}} is neither bounded, nor unbounded (in Beam's terms), but time-varying.

> Add a Beam SQL DSL
> ------------------
>                 Key: BEAM-301
>                 URL:
>             Project: Beam
>          Issue Type: New Feature
>          Components: sdk-ideas
>            Reporter: Jean-Baptiste Onofré
>            Assignee: Xu Mingmin
> The SQL DSL helps developers to build a Beam pipeline from SQL statement in String directly.

> In Phase I, it starts to support INSERT/SELECT queries with FILTERs, one example SQL
as below:
> {code}
> {code}
> A design doc is available at

This message was sent by Atlassian JIRA

View raw message