beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <>
Subject [jira] [Work logged] (BEAM-4605) Make GroupByKey the primitive in the Python SDK
Date Thu, 21 Jun 2018 00:36:00 GMT


ASF GitHub Bot logged work on BEAM-4605:

                Author: ASF GitHub Bot
            Created on: 21/Jun/18 00:35
            Start Date: 21/Jun/18 00:35
    Worklog Time Spent: 10m 
      Work Description: robertwb commented on a change in pull request #5702: [BEAM-4605]
Remove subtransform of GBK.

 File path: sdks/python/apache_beam/runners/portability/
 @@ -90,6 +91,15 @@ def run_pipeline(self, pipeline):
           pcoll.coder_id = proto_context.coders.get_id(coder)
+    # Some runners won't detect the GroupByKey transform unless it has no
+    # subtransforms.  Remove all sub-transforms until BEAM-4605 is resolved.
+    for transform_id, transform_proto in list(
+        proto_pipeline.components.transforms.items()):
+      if transform_proto.spec.urn == common_urns.primitives.GROUP_BY_KEY.urn:
+        for sub_transform in transform_proto.subtransforms:
 Review comment:
   I think logging this would be more noise than help. I hope to resolve BEAM-4605 soon, just
probably not in time for the demo. 

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:

Issue Time Tracking

    Worklog Id:     (was: 113983)
    Time Spent: 40m  (was: 0.5h)

> Make GroupByKey the primitive in the Python SDK
> -----------------------------------------------
>                 Key: BEAM-4605
>                 URL:
>             Project: Beam
>          Issue Type: Task
>          Components: sdk-py-core
>            Reporter: Robert Bradshaw
>            Assignee: Ahmet Altay
>            Priority: Major
>          Time Spent: 40m
>  Remaining Estimate: 0h
> Currently it is still a composite, with GBKOnly as a primitive. 

This message was sent by Atlassian JIRA

View raw message