beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Work logged] (BEAM-4575) Beam SQL should cleanly transform graph from Calcite
Date Tue, 19 Jun 2018 20:58:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-4575?focusedWorklogId=113377&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-113377
]

ASF GitHub Bot logged work on BEAM-4575:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 19/Jun/18 20:57
            Start Date: 19/Jun/18 20:57
    Worklog Time Spent: 10m 
      Work Description: akedin commented on a change in pull request #5687: [BEAM-4575][SQL]
Don't wait on Unbounded PCollections
URL: https://github.com/apache/beam/pull/5687#discussion_r196575927
 
 

 ##########
 File path: sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamEnumerableConverter.java
 ##########
 @@ -102,15 +107,37 @@ public Result implement(EnumerableRelImplementor implementor, Prefer
prefer) {
     }
   }
 
-  private static PipelineResult run(
+  private static @Nullable PipelineResult run(
       PipelineOptions options, BeamRelNode node, DoFn<Row, Void> doFn) {
     Pipeline pipeline = Pipeline.create(options);
     PCollectionTuple.empty(pipeline).apply(node.toPTransform()).apply(ParDo.of(doFn));
     PipelineResult result = pipeline.run();
+
+    if (containsUnboundedPCollection(pipeline)) {
+      return null;
 
 Review comment:
   But there's `pipelineResult.getState()` which you can check whether it is done yet, can't
it be used for the same purpose?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 113377)
    Time Spent: 3h  (was: 2h 50m)

> Beam SQL should cleanly transform graph from Calcite
> ----------------------------------------------------
>
>                 Key: BEAM-4575
>                 URL: https://issues.apache.org/jira/browse/BEAM-4575
>             Project: Beam
>          Issue Type: New Feature
>          Components: dsl-sql
>            Reporter: Andrew Pilloud
>            Assignee: Andrew Pilloud
>            Priority: Major
>          Time Spent: 3h
>  Remaining Estimate: 0h
>
> It would be nice if the Beam graph matched the Calcite graph in structure with each node
generating a PTransform that is applied onto the PCollection of it's parent. We should also
ensure that each Calcite node only appears in the Beam graph one time.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message