beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <>
Subject [jira] [Work logged] (BEAM-4575) Beam SQL should cleanly transform graph from Calcite
Date Tue, 19 Jun 2018 18:47:00 GMT


ASF GitHub Bot logged work on BEAM-4575:

                Author: ASF GitHub Bot
            Created on: 19/Jun/18 18:46
            Start Date: 19/Jun/18 18:46
    Worklog Time Spent: 10m 
      Work Description: akedin commented on a change in pull request #5673: [BEAM-4575] Cleanly
transform graph from Calcite to Beam SQL

 File path: sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/
 @@ -102,10 +102,9 @@ public void registerUdaf(String functionName, Combine.CombineFn combineFn)
     defaultSchema.add(functionName, new UdafImpl(combineFn));
-  public PTransform<PCollectionTuple, PCollection<Row>> parseQuery(String query)
-      throws ParseException {
+  public PCollection<Row> parseQuery(Pipeline pipeline, String query) throws ParseException
     try {
-      return planner.convertToBeamRel(query).toPTransform();
+      return BeamSqlRelUtils.toPCollection(pipeline, planner.convertToBeamRel(query));
 Review comment:
   Why does parser need to know about the pipeline? Can this logic happen outside of `BeamSqlEnv`?
I would rather have another abstraction returned from `parseQuery` instead of passing a mutable
global state into some blackbox. For example, I am thinking we can return something like `SqlTransform`
here, if we could construct it with an instance of current `BeamSqlEnv`

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:

Issue Time Tracking

    Worklog Id:     (was: 113318)
    Time Spent: 1.5h  (was: 1h 20m)

> Beam SQL should cleanly transform graph from Calcite
> ----------------------------------------------------
>                 Key: BEAM-4575
>                 URL:
>             Project: Beam
>          Issue Type: New Feature
>          Components: dsl-sql
>            Reporter: Andrew Pilloud
>            Assignee: Andrew Pilloud
>            Priority: Major
>          Time Spent: 1.5h
>  Remaining Estimate: 0h
> It would be nice if the Beam graph matched the Calcite graph in structure with each node
generating a PTransform that is applied onto the PCollection of it's parent. We should also
ensure that each Calcite node only appears in the Beam graph one time.

This message was sent by Atlassian JIRA

View raw message