beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From XuMingmin <>
Subject [GitHub] beam pull request #3250: [Beam-2010] expose programming interface
Date Sat, 27 May 2017 22:08:13 GMT
GitHub user XuMingmin opened a pull request:

    [Beam-2010] expose programming interface

    It's the DSL implementation described in [Apache Beam: design of DSL SQL interface](

    It provides two methods `BeamSql.simpleQuery(...)` and `BeamSql.query(...)` to integrate
SQL in Beam pipeline, by converting it into a `PTransform`, refer to `org.apache.beam.dsls.sql.example.BeamSqlExample`
as a quick example.
    Along with this DSL methods, some functions in the back-end are changed:
    1. `BeamSqlRowCoder` is updated to have a input parameter `BeamSQLRecordType`, to declare
row format, and it cannot be registered during initialization any more;
    2. Interface in `BeamRelNode` is changed, to align with the goal, converting into PTransform;
    The methods for CLI client are marked as `Experimental` in `BeamSql`, the difference is
handled in `BeamIOSourceRel`, that whether an upstream PCollection is there, or need to generate
from `BaseBeamTable`;

You can merge this pull request into a Git repository by running:

    $ git pull BEAM-2010

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3250
commit 3f0c2c356a83163f165684e07b24bbcc9f759ab7
Author: mingmxu <>
Date:   2017-05-27T05:07:03Z

    DSL interface for Beam SQL


If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

View raw message